class PeelingModuloScheduleExpander

Declaration

class PeelingModuloScheduleExpander { /* full declaration omitted */ };

Description

A reimplementation of ModuloScheduleExpander. It works by generating a standalone kernel loop and peeling out the prologs and epilogs.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:280

Member Variables

protected llvm::ModuloSchedule& Schedule
protected llvm::MachineFunction& MF
protected const llvm::TargetSubtargetInfo& ST
protected llvm::MachineRegisterInfo& MRI
protected const llvm::TargetInstrInfo* TII
protected llvm::LiveIntervals* LIS
protected llvm::MachineBasicBlock* BB
The original loop block that gets rewritten in-place.
protected llvm::MachineBasicBlock* Preheader
The original loop preheader.
protected SmallVector<llvm::MachineBasicBlock*, 4> Prologs
All prolog and epilog blocks.
protected SmallVector<llvm::MachineBasicBlock*, 4> Epilogs
All prolog and epilog blocks.
protected DenseMap<llvm::MachineBasicBlock*, llvm::BitVector> LiveStages
For every block, the stages that are produced.
protected DenseMap<llvm::MachineBasicBlock*, llvm::BitVector> AvailableStages
For every block, the stages that are available. A stage can be available but not produced (in the epilog) or produced but not available (in the prolog).
protected DenseMap<llvm::MachineInstr*, unsigned int> PhiNodeLoopIteration
When peeling the epilogue keep track of the distance between the phi nodes and the kernel.
protected DenseMap<llvm::MachineInstr*, llvm::MachineInstr*> CanonicalMIs
CanonicalMIs and BlockMIs form a bidirectional map between any of the loop kernel clones.
protected DenseMap< std::pair<MachineBasicBlock*, MachineInstr*>, llvm::MachineInstr*> BlockMIs
protected std::deque<MachineBasicBlock*> PeeledFront
State passed from peelKernel to peelPrologAndEpilogs().
protected std::deque<MachineBasicBlock*> PeeledBack
State passed from peelKernel to peelPrologAndEpilogs().
protected SmallVector<llvm::MachineInstr*, 4> IllegalPhisToDelete
Illegal phis that need to be deleted once we re-link stages.
protected std::unique_ptr< TargetInstrInfo::PipelinerLoopInfo> LoopInfo
Target loop info before kernel peeling.

Method Overview

Methods

llvm::MachineBasicBlock* CreateLCSSAExitingBlock()

Description

Create a poor-man's LCSSA by cloning only the PHIs from the kernel block to a block dominated by all prologs and epilogs. This allows us to treat the loop exiting block as any other kernel clone.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:356

PeelingModuloScheduleExpander(
    llvm::MachineFunction& MF,
    llvm::ModuloSchedule& S,
    llvm::LiveIntervals* LIS)

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:282

Parameters

llvm::MachineFunction& MF
llvm::ModuloSchedule& S
llvm::LiveIntervals* LIS

void expand()

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:287

void filterInstructions(
    llvm::MachineBasicBlock* MB,
    int MinStage)

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:337

Parameters

llvm::MachineBasicBlock* MB
int MinStage

void fixupBranches()

Description

Insert branches between prologs, kernel and epilogs.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:352

llvm::Register getEquivalentRegisterIn(
    llvm::Register Reg,
    llvm::MachineBasicBlock* BB)

Description

All prolog and epilog blocks are clones of the kernel, so any produced register in one block has an corollary in all other blocks.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:347

Parameters

llvm::Register Reg
llvm::MachineBasicBlock* BB

llvm::Register getPhiCanonicalReg(
    llvm::MachineInstr* CanonicalPhi,
    llvm::MachineInstr* Phi)

Description

Helper function to find the right canonical register for a phi instruction coming from a peeled out prologue.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:365

Parameters

llvm::MachineInstr* CanonicalPhi
llvm::MachineInstr* Phi

unsigned int getStage(llvm::MachineInstr* MI)

Description

Helper to get the stage of an instruction in the schedule.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:358

Parameters

llvm::MachineInstr* MI

void moveStageBetweenBlocks(
    llvm::MachineBasicBlock* DestBB,
    llvm::MachineBasicBlock* SourceBB,
    unsigned int Stage)

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:340

Parameters

llvm::MachineBasicBlock* DestBB
llvm::MachineBasicBlock* SourceBB
unsigned int Stage

llvm::MachineBasicBlock* peelKernel(
    llvm::LoopPeelDirection LPD)

Description

Peels one iteration of the rewritten kernel (BB) in the specified direction.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:334

Parameters

llvm::LoopPeelDirection LPD

void peelPrologAndEpilogs()

Description

Peel the kernel forwards and backwards to produce prologs and epilogs, and stitch them together.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:344

void rewriteKernel()

Description

Converts BB from the original loop body to the rewritten, pipelined steady-state.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:330

void rewriteUsesOf(llvm::MachineInstr* MI)

Description

Change all users of MI, if MI is predicated out (LiveStages[MI->getParent()] == false).

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:350

Parameters

llvm::MachineInstr* MI

void validateAgainstModuloScheduleExpander()

Description

Runs ModuloScheduleExpander and treats it as a golden input to validate aspects of the code generated by PeelingModuloScheduleExpander.

Declared at: llvm/include/llvm/CodeGen/ModuloSchedule.h:291