class InterleavedAccessInfo

Declaration

class InterleavedAccessInfo { /* full declaration omitted */ };

Description

Drive the analysis of interleaved memory accesses in the loop. Use this class to analyze interleaved accesses only when we can vectorize a loop. Otherwise it's meaningless to do analysis as the vectorization on interleaved accesses is unsafe. The analysis collects interleave groups and records the relationships between the member and the group in a map.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:751

Member Variables

private llvm::PredicatedScalarEvolution& PSE
A wrapper around ScalarEvolution, used to add runtime SCEV checks. Simplifies SCEV expressions in the context of existing SCEV assumptions. The interleaved access analysis can also add new predicates (for example by versioning strides of pointers).
private llvm::Loop* TheLoop
private llvm::DominatorTree* DT
private llvm::LoopInfo* LI
private const llvm::LoopAccessInfo* LAI
private bool RequiresScalarEpilogue = false
True if the loop may contain non-reversed interleaved groups with out-of-bounds accesses. We ensure we don't speculatively access memory out-of-bounds by executing at least one scalar epilogue iteration.
private DenseMap<llvm::Instruction*, InterleaveGroup<llvm::Instruction>*> InterleaveGroupMap
Holds the relationships between the members and the interleave group.
private SmallPtrSet<InterleaveGroup<llvm::Instruction>*, 4> InterleaveGroups
private DenseMap<llvm::Instruction*, SmallPtrSet<llvm::Instruction*, 2>> Dependences
Holds dependences among the memory accesses in the loop. It maps a source access to a set of dependent sink accesses.

Method Overview

Methods

InterleavedAccessInfo(
    llvm::PredicatedScalarEvolution& PSE,
    llvm::Loop* L,
    llvm::DominatorTree* DT,
    llvm::LoopInfo* LI,
    const llvm::LoopAccessInfo* LAI)

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:753

Parameters

llvm::PredicatedScalarEvolution& PSE
llvm::Loop* L
llvm::DominatorTree* DT
llvm::LoopInfo* LI
const llvm::LoopAccessInfo* LAI

void analyzeInterleaving(
    bool EnableMaskedInterleavedGroup)

Description

Analyze the interleaved accesses and collect them in interleave groups. Substitute symbolic strides using \p Strides. Consider also predicated loads/stores in the analysis if\p EnableMaskedInterleavedGroup is true.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:764

Parameters

bool EnableMaskedInterleavedGroup

bool areDependencesValid() const

Description

Returns true if LoopAccessInfo can be used for dependence queries.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:901

bool canReorderMemAccessesForInterleavedGroups(
    llvm::InterleavedAccessInfo::StrideEntry* A,
    llvm::InterleavedAccessInfo::StrideEntry* B)
    const

Description

Returns true if memory accesses \p A and \p B can be reordered, if necessary, when constructing interleaved groups. \p A must precede \p B in program order. We return false if reordering is not necessary or is prevented because \p A and \p B may be dependent.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:910

Parameters

llvm::InterleavedAccessInfo::StrideEntry* A
llvm::InterleavedAccessInfo::StrideEntry* B

void collectConstStrideAccesses(
    MapVector<llvm::Instruction*,
              llvm::InterleavedAccessInfo::
                  StrideDescriptor>&
        AccessStrideInfo,
    const llvm::ValueToValueMap& Strides)

Description

Collect all the accesses with a constant stride in program order.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:888

Parameters

MapVector<llvm::Instruction*, llvm::InterleavedAccessInfo:: StrideDescriptor>& AccessStrideInfo
const llvm::ValueToValueMap& Strides

void collectDependences()

Description

Collect the dependences from LoopAccessInfo. We process the dependences once during the interleaved access analysis to enable constant-time dependence queries.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:958

InterleaveGroup<llvm::Instruction>*
createInterleaveGroup(llvm::Instruction* Instr,
                      int Stride,
                      llvm::Align Alignment)

Description

Create a new interleave group with the given instruction \p Instr, stride \p Stride and alignment \p Align.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:868

Parameters

llvm::Instruction* Instr
int Stride
llvm::Align Alignment

Returns

the newly created interleave group.

InterleaveGroup<llvm::Instruction>*
getInterleaveGroup(
    const llvm::Instruction* Instr) const

Description

Get the interleave group that \p Instr belongs to.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:796

Parameters

const llvm::Instruction* Instr

Returns

nullptr if doesn't have such group.

iterator_range<SmallPtrSetIterator<
    llvm::InterleaveGroup<Instruction>*>>
getInterleaveGroups()

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:801

bool invalidateGroups()

Description

Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption. Although we currently prevent group formation for predicated accesses, we may be able to relax this limitation in the future once we handle more complicated blocks. Returns true if any groups were invalidated.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:771

void invalidateGroupsRequiringScalarEpilogue()

Description

Invalidate groups that require a scalar epilogue (due to gaps). This can happen when optimizing for size forbids a scalar epilogue, and the gap cannot be filtered by masking the load/store.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:812

bool isInterleaved(llvm::Instruction* Instr) const

Description

Check if \p Instr belongs to any interleave group.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:788

Parameters

llvm::Instruction* Instr

bool isPredicated(llvm::BasicBlock* BB) const

Description

Returns true if \p BB is a predicated block.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:896

Parameters

llvm::BasicBlock* BB

static bool isStrided(int Stride)

Description

Returns true if \p Stride is allowed in an interleaved group.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:893

Parameters

int Stride

void releaseGroup(
    InterleaveGroup<llvm::Instruction>* Group)

Description

Release the group and remove all the relationships.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:878

Parameters

InterleaveGroup<llvm::Instruction>* Group

bool requiresScalarEpilogue() const

Description

Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:807

~InterleavedAccessInfo()

Declared at: llvm/include/llvm/Analysis/VectorUtils.h:758