struct BoUpSLP::BlockScheduling

Declaration

struct BoUpSLP::BlockScheduling { /* full declaration omitted */ };

Description

Contains all scheduling data for a basic block. It does not schedules instructions, which are not memory read/write instructions and their operands are either constants, or arguments, or phis, or instructions from others blocks, or their users are phis or from the other blocks. The resulting vector instructions can be placed at the beginning of the basic block without scheduling (if operands does not need to be scheduled) or at the end of the block (if users are outside of the block). It allows to save some compile time and memory used by the compiler. ScheduleData is assigned for each instruction in between the boundaries of the tree entry, even for those, which are not part of the graph. It is required to correctly follow the dependencies between the instructions and their correct scheduling. The ScheduleData is not allocated for the instructions, which do not require scheduling, like phis, nodes with extractelements/insertelements only or nodes with instructions, with uses/operands outside of the block.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2892

Member Variables

public llvm::BasicBlock* BB

public std::vector<std::unique_ptr<ScheduleData[]>> ScheduleDataChunks

Simple memory allocation for ScheduleData.

public int ChunkSize

The size of a ScheduleData array in ScheduleDataChunks.

public int ChunkPos

The allocator position in the current chunk, which is the last entry of ScheduleDataChunks.

public DenseMap< llvm::Instruction*, llvm::slpvectorizer::BoUpSLP::ScheduleData*> ScheduleDataMap

Attaches ScheduleData to Instruction. Note that the mapping survives during all vectorization iterations, i.e. ScheduleData structures are recycled.

public DenseMap< llvm::Value*, SmallDenseMap<llvm::Value*, llvm::slpvectorizer::BoUpSLP:: ScheduleData*>> ExtraScheduleDataMap

Attaches ScheduleData to Instruction with the leading key.

public SetVector< llvm::slpvectorizer::BoUpSLP::ScheduleData*> ReadyInsts

The ready-list for scheduling (only used for the dry-run).

public llvm::Instruction* ScheduleStart = nullptr

The first instruction of the scheduling region.

public llvm::Instruction* ScheduleEnd = nullptr

The first instruction _after_ the scheduling region.

public llvm::slpvectorizer::BoUpSLP::ScheduleData* FirstLoadStoreInRegion = nullptr

The first memory accessing instruction in the scheduling region (can be null).

public llvm::slpvectorizer::BoUpSLP::ScheduleData* LastLoadStoreInRegion = nullptr

The last memory accessing instruction in the scheduling region (can be null).

public bool RegionHasStackSave = false

Is there an llvm.stacksave or llvm.stackrestore in the scheduling region? Used to optimize the dependence calculation for the common case where there isn't.

public int ScheduleRegionSize = 0

The current size of the scheduling region.

public int ScheduleRegionSizeLimit = ScheduleRegionSizeBudget

The maximum size allowed for the scheduling region.

public int SchedulingRegionID = 1

The ID of the scheduling region. For a new vectorization iteration this is incremented which "removes" all ScheduleData from the region. Make sure that the initial SchedulingRegionID is greater than the initial SchedulingRegionID in ScheduleData (which is 0).

Method Overview

• public BlockScheduling(llvm::BasicBlock * BB)
• public llvm::slpvectorizer::BoUpSLP::ScheduleData * allocateScheduleDataChunks()
• public llvm::slpvectorizer::BoUpSLP::ScheduleData * buildBundle(ArrayRef<llvm::Value *> VL)
• public void calculateDependencies(llvm::slpvectorizer::BoUpSLP::ScheduleData * SD, bool InsertInReadyList, llvm::slpvectorizer::BoUpSLP * SLP)
• public void cancelScheduling(ArrayRef<llvm::Value *> VL, llvm::Value * OpValue)
• public void clear()
• public void doForAllOpcodes(llvm::Value * V, function_ref<void (llvm::slpvectorizer::BoUpSLP::ScheduleData *)> Action)
• public bool extendSchedulingRegion(llvm::Value * V, const (anonymous namespace)::InstructionsState & S)
• public llvm::slpvectorizer::BoUpSLP::ScheduleData * getScheduleData(llvm::Instruction * I)
• public llvm::slpvectorizer::BoUpSLP::ScheduleData * getScheduleData(llvm::Value * V)
• public llvm::slpvectorizer::BoUpSLP::ScheduleData * getScheduleData(llvm::Value * V, llvm::Value * Key)
• public void initScheduleData(llvm::Instruction * FromI, llvm::Instruction * ToI, llvm::slpvectorizer::BoUpSLP::ScheduleData * PrevLoadStore, llvm::slpvectorizer::BoUpSLP::ScheduleData * NextLoadStore)
• public template <typename ReadyListType>void initialFillReadyList(ReadyListType & ReadyList)
• public bool isInSchedulingRegion(llvm::slpvectorizer::BoUpSLP::ScheduleData * SD) const
• public void resetSchedule()
• public template <typename ReadyListType>void schedule(llvm::slpvectorizer::BoUpSLP::ScheduleData * SD, ReadyListType & ReadyList)
• public Optional<llvm::slpvectorizer::BoUpSLP::ScheduleData *> tryScheduleBundle(ArrayRef<llvm::Value *> VL, llvm::slpvectorizer::BoUpSLP * SLP, const (anonymous namespace)::InstructionsState & S)
• public void verify()

Methods

¶BlockScheduling(llvm::BasicBlock* BB)

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2893

Parameters

llvm::BasicBlock* BB

¶llvm::slpvectorizer::BoUpSLP::ScheduleData*
allocateScheduleDataChunks()

Description

Allocates schedule data chunk.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3115

¶llvm::slpvectorizer::BoUpSLP::ScheduleData*
buildBundle(ArrayRef<llvm::Value*> VL)

Description

Build a bundle from the ScheduleData nodes corresponding to the scalar instruction for each lane.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3100

Parameters

ArrayRef<llvm::Value*> VL

¶void calculateDependencies(
    llvm::slpvectorizer::BoUpSLP::ScheduleData*
        SD,
    bool InsertInReadyList,
    llvm::slpvectorizer::BoUpSLP* SLP)

Description

Updates the dependency information of a bundle and of all instructions/ bundles which depend on the original bundle.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3129

Parameters

llvm::slpvectorizer::BoUpSLP::ScheduleData* SD

bool InsertInReadyList

llvm::slpvectorizer::BoUpSLP* SLP

¶void cancelScheduling(ArrayRef<llvm::Value*> VL,
                      llvm::Value* OpValue)

Description

Un-bundles a group of instructions.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3112

Parameters

ArrayRef<llvm::Value*> VL

llvm::Value* OpValue

¶void clear()

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2896

¶void doForAllOpcodes(
    llvm::Value* V,
    function_ref<void(llvm::slpvectorizer::
                          BoUpSLP::ScheduleData*)>
        Action)

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3072

Parameters

llvm::Value* V

function_ref<void( llvm::slpvectorizer::BoUpSLP::ScheduleData*)> Action

¶bool extendSchedulingRegion(
    llvm::Value* V,
    const(anonymous namespace)::InstructionsState&
        S)

Description

Extends the scheduling region so that V is inside the region.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3119

Parameters

llvm::Value* V

const(anonymous namespace)::InstructionsState& S

Returns

true if the region size is within the limit.

¶llvm::slpvectorizer::BoUpSLP::ScheduleData*
getScheduleData(llvm::Instruction* I)

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2916

Parameters

llvm::Instruction* I

¶llvm::slpvectorizer::BoUpSLP::ScheduleData*
getScheduleData(llvm::Value* V)

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2926

Parameters

llvm::Value* V

¶llvm::slpvectorizer::BoUpSLP::ScheduleData*
getScheduleData(llvm::Value* V, llvm::Value* Key)

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2932

Parameters

llvm::Value* V

llvm::Value* Key

¶void initScheduleData(
    llvm::Instruction* FromI,
    llvm::Instruction* ToI,
    llvm::slpvectorizer::BoUpSLP::ScheduleData*
        PrevLoadStore,
    llvm::slpvectorizer::BoUpSLP::ScheduleData*
        NextLoadStore)

Description

Initialize the ScheduleData structures for new instructions in the scheduling region.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3123

Parameters

llvm::Instruction* FromI

llvm::Instruction* ToI

llvm::slpvectorizer::BoUpSLP::ScheduleData* PrevLoadStore

llvm::slpvectorizer::BoUpSLP::ScheduleData* NextLoadStore

¶template <typename ReadyListType>
void initialFillReadyList(
    ReadyListType& ReadyList)

Description

Put all instructions into the ReadyList which are ready for scheduling.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3085

Templates

ReadyListType

Parameters

ReadyListType& ReadyList

¶bool isInSchedulingRegion(
    llvm::slpvectorizer::BoUpSLP::ScheduleData*
        SD) const

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2944

Parameters

llvm::slpvectorizer::BoUpSLP::ScheduleData* SD

¶void resetSchedule()

Description

Sets all instruction in the scheduling region to un-scheduled.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3133

¶template <typename ReadyListType>
void schedule(
    llvm::slpvectorizer::BoUpSLP::ScheduleData*
        SD,
    ReadyListType& ReadyList)

Description

Marks an instruction as scheduled and puts all dependent ready instructions into the ready-list.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:2951

Templates

ReadyListType

Parameters

llvm::slpvectorizer::BoUpSLP::ScheduleData* SD

ReadyListType& ReadyList

¶Optional<
    llvm::slpvectorizer::BoUpSLP::ScheduleData*>
tryScheduleBundle(
    ArrayRef<llvm::Value*> VL,
    llvm::slpvectorizer::BoUpSLP* SLP,
    const(anonymous namespace)::InstructionsState&
        S)

Description

Checks if a bundle of instructions can be scheduled, i.e. has no cyclic dependencies. This is only a dry-run, no instructions are actually moved at this stage.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3108

Parameters

ArrayRef<llvm::Value*> VL

llvm::slpvectorizer::BoUpSLP* SLP

const(anonymous namespace)::InstructionsState& S

Returns

the scheduling bundle. The returned Optional value is non-None if \p VL is allowed to be scheduled.

¶void verify()

Description

Verify basic self consistency properties of the data structure.

Declared at: llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp:3045