class EpilogueVectorizerEpilogueLoop
Declaration
class EpilogueVectorizerEpilogueLoop
: public InnerLoopAndEpilogueVectorizer { /* full declaration omitted */ };
Description
An extension of the inner loop vectorizer that creates a skeleton for a vectorized loop that has its epilogue (residual) also vectorized. The idea is to run the vplan on a given loop twice, firstly to setup the skeleton and vectorize the main loop, and secondly to complete the skeleton from the first step and vectorize the epilogue. This is achieved by deriving two concrete strategy classes from this base class and invoking them in succession from the loop vectorizer planner.
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:851
Inherits from: InnerLoopAndEpilogueVectorizer
Member Variables
Inherited from InnerLoopAndEpilogueVectorizer:
Inherited from InnerLoopVectorizer:
- protected OrigLoop
- protected PSE
- protected LI
- protected DT
- protected AA
- protected TLI
- protected TTI
- protected AC
- protected ORE
- protected VF
- protected MinProfitableTripCount
- protected UF
- protected Builder
- protected LoopVectorPreHeader
- protected LoopScalarPreHeader
- protected LoopMiddleBlock
- protected LoopExitBlock
- protected LoopScalarBody
- protected LoopBypassBlocks
- protected PredicatedInstructions
- protected TripCount = nullptr
- protected VectorTripCount = nullptr
- protected Legal
- protected Cost
- protected AddedSafetyChecks = false
- protected IVEndValues
- protected BFI
- protected PSI
- protected OptForSizeBasedOnProfile
- protected RTChecks
- protected ReductionResumeValues
Method Overview
- public EpilogueVectorizerEpilogueLoop(llvm::Loop * OrigLoop, llvm::PredicatedScalarEvolution & PSE, llvm::LoopInfo * LI, llvm::DominatorTree * DT, const llvm::TargetLibraryInfo * TLI, const llvm::TargetTransformInfo * TTI, llvm::AssumptionCache * AC, llvm::OptimizationRemarkEmitter * ORE, llvm::EpilogueLoopVectorizationInfo & EPI, llvm::LoopVectorizationLegality * LVL, llvm::LoopVectorizationCostModel * CM, llvm::BlockFrequencyInfo * BFI, llvm::ProfileSummaryInfo * PSI, GeneratedRTChecks & Checks)
- public std::pair<BasicBlock *, Value *> createEpilogueVectorizedLoopSkeleton()
- protected llvm::BasicBlock * emitMinimumVectorEpilogueIterCountCheck(llvm::BasicBlock * Bypass, llvm::BasicBlock * Insert)
- protected void printDebugTracesAtEnd()
- protected void printDebugTracesAtStart()
Inherited from InnerLoopAndEpilogueVectorizer:
Inherited from InnerLoopVectorizer:
- public areSafetyChecksAdded
- protected clearReductionWrapFlags
- protected collectPoisonGeneratingRecipes
- protected completeLoopSkeleton
- protected createBitOrPointerCast
- protected createInductionResumeValues
- protected createVectorLoopSkeleton
- public createVectorizedLoopSkeleton
- protected emitIterationCountCheck
- protected emitMemRuntimeChecks
- protected emitSCEVChecks
- protected fixCrossIterationPHIs
- protected fixFirstOrderRecurrence
- public fixNonInductionPHIs
- protected fixReduction
- public fixVectorizedLoop
- protected fixupIVUsers
- public getBroadcastInstrs
- protected getOrCreateTripCount
- protected getOrCreateVectorTripCount
- public getReductionResumeValue
- public packScalarIntoVectorValue
- protected printDebugTracesAtEnd
- protected printDebugTracesAtStart
- public scalarizeInstruction
- protected sinkScalarOperands
- protected truncateToMinimalBitwidths
- public useOrderedReductions
- public vectorizeInterleaveGroup
- public widenCallInstruction
Methods
¶EpilogueVectorizerEpilogueLoop(
llvm::Loop* OrigLoop,
llvm::PredicatedScalarEvolution& PSE,
llvm::LoopInfo* LI,
llvm::DominatorTree* DT,
const llvm::TargetLibraryInfo* TLI,
const llvm::TargetTransformInfo* TTI,
llvm::AssumptionCache* AC,
llvm::OptimizationRemarkEmitter* ORE,
llvm::EpilogueLoopVectorizationInfo& EPI,
llvm::LoopVectorizationLegality* LVL,
llvm::LoopVectorizationCostModel* CM,
llvm::BlockFrequencyInfo* BFI,
llvm::ProfileSummaryInfo* PSI,
GeneratedRTChecks& Checks)
EpilogueVectorizerEpilogueLoop(
llvm::Loop* OrigLoop,
llvm::PredicatedScalarEvolution& PSE,
llvm::LoopInfo* LI,
llvm::DominatorTree* DT,
const llvm::TargetLibraryInfo* TLI,
const llvm::TargetTransformInfo* TTI,
llvm::AssumptionCache* AC,
llvm::OptimizationRemarkEmitter* ORE,
llvm::EpilogueLoopVectorizationInfo& EPI,
llvm::LoopVectorizationLegality* LVL,
llvm::LoopVectorizationCostModel* CM,
llvm::BlockFrequencyInfo* BFI,
llvm::ProfileSummaryInfo* PSI,
GeneratedRTChecks& Checks)
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:853
Parameters
- llvm::Loop* OrigLoop
- llvm::PredicatedScalarEvolution& PSE
- llvm::LoopInfo* LI
- llvm::DominatorTree* DT
- const llvm::TargetLibraryInfo* TLI
- const llvm::TargetTransformInfo* TTI
- llvm::AssumptionCache* AC
- llvm::OptimizationRemarkEmitter* ORE
- llvm::EpilogueLoopVectorizationInfo& EPI
- llvm::LoopVectorizationLegality* LVL
- llvm::LoopVectorizationCostModel* CM
- llvm::BlockFrequencyInfo* BFI
- llvm::ProfileSummaryInfo* PSI
- GeneratedRTChecks& Checks
¶std::pair<BasicBlock*, Value*>
createEpilogueVectorizedLoopSkeleton()
std::pair<BasicBlock*, Value*>
createEpilogueVectorizedLoopSkeleton()
Description
Implements the interface for creating a vectorized skeleton using the *epilogue loop* strategy (ie the second pass of vplan execution).
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:867
¶llvm::BasicBlock*
emitMinimumVectorEpilogueIterCountCheck(
llvm::BasicBlock* Bypass,
llvm::BasicBlock* Insert)
llvm::BasicBlock*
emitMinimumVectorEpilogueIterCountCheck(
llvm::BasicBlock* Bypass,
llvm::BasicBlock* Insert)
Description
Emits an iteration count bypass check after the main vector loop has finished to see if there are any iterations left to execute by either the vector epilogue or the scalar epilogue.
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:873
Parameters
- llvm::BasicBlock* Bypass
- llvm::BasicBlock* Insert
¶void printDebugTracesAtEnd()
void printDebugTracesAtEnd()
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:877
¶void printDebugTracesAtStart()
void printDebugTracesAtStart()
Description
Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
Declared at: llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:876