class PeelingModuloScheduleExpander
Declaration
class PeelingModuloScheduleExpander { /* full declaration omitted */ };
Description
A reimplementation of ModuloScheduleExpander. It works by generating a standalone kernel loop and peeling out the prologs and epilogs.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:280
Member Variables
- protected llvm::ModuloSchedule& Schedule
- protected llvm::MachineFunction& MF
- protected const llvm::TargetSubtargetInfo& ST
- protected llvm::MachineRegisterInfo& MRI
- protected const llvm::TargetInstrInfo* TII
- protected llvm::LiveIntervals* LIS
- protected llvm::MachineBasicBlock* BB
- The original loop block that gets rewritten in-place.
- protected llvm::MachineBasicBlock* Preheader
- The original loop preheader.
- protected SmallVector<llvm::MachineBasicBlock*, 4> Prologs
- All prolog and epilog blocks.
- protected SmallVector<llvm::MachineBasicBlock*, 4> Epilogs
- All prolog and epilog blocks.
- protected DenseMap<llvm::MachineBasicBlock*, llvm::BitVector> LiveStages
- For every block, the stages that are produced.
- protected DenseMap<llvm::MachineBasicBlock*, llvm::BitVector> AvailableStages
- For every block, the stages that are available. A stage can be available but not produced (in the epilog) or produced but not available (in the prolog).
- protected DenseMap<llvm::MachineInstr*, unsigned int> PhiNodeLoopIteration
- When peeling the epilogue keep track of the distance between the phi nodes and the kernel.
- protected DenseMap<llvm::MachineInstr*, llvm::MachineInstr*> CanonicalMIs
- CanonicalMIs and BlockMIs form a bidirectional map between any of the loop kernel clones.
- protected DenseMap< std::pair<MachineBasicBlock*, MachineInstr*>, llvm::MachineInstr*> BlockMIs
- protected std::deque<MachineBasicBlock*> PeeledFront
- State passed from peelKernel to peelPrologAndEpilogs().
- protected std::deque<MachineBasicBlock*> PeeledBack
- State passed from peelKernel to peelPrologAndEpilogs().
- protected SmallVector<llvm::MachineInstr*, 4> IllegalPhisToDelete
- Illegal phis that need to be deleted once we re-link stages.
- protected std::unique_ptr< TargetInstrInfo::PipelinerLoopInfo> LoopInfo
- Target loop info before kernel peeling.
Method Overview
- protected llvm::MachineBasicBlock * CreateLCSSAExitingBlock()
- public PeelingModuloScheduleExpander(llvm::MachineFunction & MF, llvm::ModuloSchedule & S, llvm::LiveIntervals * LIS)
- public void expand()
- protected void filterInstructions(llvm::MachineBasicBlock * MB, int MinStage)
- protected void fixupBranches()
- protected llvm::Register getEquivalentRegisterIn(llvm::Register Reg, llvm::MachineBasicBlock * BB)
- protected llvm::Register getPhiCanonicalReg(llvm::MachineInstr * CanonicalPhi, llvm::MachineInstr * Phi)
- protected unsigned int getStage(llvm::MachineInstr * MI)
- protected void moveStageBetweenBlocks(llvm::MachineBasicBlock * DestBB, llvm::MachineBasicBlock * SourceBB, unsigned int Stage)
- protected llvm::MachineBasicBlock * peelKernel(llvm::LoopPeelDirection LPD)
- protected void peelPrologAndEpilogs()
- protected void rewriteKernel()
- protected void rewriteUsesOf(llvm::MachineInstr * MI)
- public void validateAgainstModuloScheduleExpander()
Methods
¶llvm::MachineBasicBlock* CreateLCSSAExitingBlock()
llvm::MachineBasicBlock* CreateLCSSAExitingBlock()
Description
Create a poor-man's LCSSA by cloning only the PHIs from the kernel block to a block dominated by all prologs and epilogs. This allows us to treat the loop exiting block as any other kernel clone.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:356
¶PeelingModuloScheduleExpander(
llvm::MachineFunction& MF,
llvm::ModuloSchedule& S,
llvm::LiveIntervals* LIS)
PeelingModuloScheduleExpander(
llvm::MachineFunction& MF,
llvm::ModuloSchedule& S,
llvm::LiveIntervals* LIS)
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:282
Parameters
¶void expand()
void expand()
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:287
¶void filterInstructions(
llvm::MachineBasicBlock* MB,
int MinStage)
void filterInstructions(
llvm::MachineBasicBlock* MB,
int MinStage)
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:337
Parameters
- llvm::MachineBasicBlock* MB
- int MinStage
¶void fixupBranches()
void fixupBranches()
Description
Insert branches between prologs, kernel and epilogs.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:352
¶llvm::Register getEquivalentRegisterIn(
llvm::Register Reg,
llvm::MachineBasicBlock* BB)
llvm::Register getEquivalentRegisterIn(
llvm::Register Reg,
llvm::MachineBasicBlock* BB)
Description
All prolog and epilog blocks are clones of the kernel, so any produced register in one block has an corollary in all other blocks.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:347
Parameters
¶llvm::Register getPhiCanonicalReg(
llvm::MachineInstr* CanonicalPhi,
llvm::MachineInstr* Phi)
llvm::Register getPhiCanonicalReg(
llvm::MachineInstr* CanonicalPhi,
llvm::MachineInstr* Phi)
Description
Helper function to find the right canonical register for a phi instruction coming from a peeled out prologue.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:365
Parameters
- llvm::MachineInstr* CanonicalPhi
- llvm::MachineInstr* Phi
¶unsigned int getStage(llvm::MachineInstr* MI)
unsigned int getStage(llvm::MachineInstr* MI)
Description
Helper to get the stage of an instruction in the schedule.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:358
Parameters
¶void moveStageBetweenBlocks(
llvm::MachineBasicBlock* DestBB,
llvm::MachineBasicBlock* SourceBB,
unsigned int Stage)
void moveStageBetweenBlocks(
llvm::MachineBasicBlock* DestBB,
llvm::MachineBasicBlock* SourceBB,
unsigned int Stage)
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:340
Parameters
- llvm::MachineBasicBlock* DestBB
- llvm::MachineBasicBlock* SourceBB
- unsigned int Stage
¶llvm::MachineBasicBlock* peelKernel(
llvm::LoopPeelDirection LPD)
llvm::MachineBasicBlock* peelKernel(
llvm::LoopPeelDirection LPD)
Description
Peels one iteration of the rewritten kernel (BB) in the specified direction.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:334
Parameters
- llvm::LoopPeelDirection LPD
¶void peelPrologAndEpilogs()
void peelPrologAndEpilogs()
Description
Peel the kernel forwards and backwards to produce prologs and epilogs, and stitch them together.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:344
¶void rewriteKernel()
void rewriteKernel()
Description
Converts BB from the original loop body to the rewritten, pipelined steady-state.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:330
¶void rewriteUsesOf(llvm::MachineInstr* MI)
void rewriteUsesOf(llvm::MachineInstr* MI)
Description
Change all users of MI, if MI is predicated out (LiveStages[MI->getParent()] == false).
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:350
Parameters
¶void validateAgainstModuloScheduleExpander()
void validateAgainstModuloScheduleExpander()
Description
Runs ModuloScheduleExpander and treats it as a golden input to validate aspects of the code generated by PeelingModuloScheduleExpander.
Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:291