struct FormatToken

Declaration

struct FormatToken { /* full declaration omitted */ };

Description

A wrapper around a \c Token storing information about the whitespace characters preceding it.

Declared at: clang/lib/Format/FormatToken.h:227

Member Variables

public clang::Token Tok
The \c Token.
public llvm::StringRef TokenText
Contains the raw token text without leading whitespace and without leading escaped newlines.
public std::shared_ptr<TokenRole> Role
A token can have a special role that can carry extra information about the token's formatting. FIXME: Make FormatToken for parsing and AnnotatedToken two different classes and make this a unique_ptr in the AnnotatedToken class.
public clang::SourceRange WhitespaceRange
The range of the whitespace immediately preceding the \c Token.
public unsigned int HasUnescapedNewline
Whether there is at least one unescaped newline before the \c Token.
public unsigned int IsMultiline
Whether the token text contains newlines (escaped or not).
public unsigned int IsFirst
Indicates that this is the first token of the file.
public unsigned int MustBreakBefore
This happens for example when a preprocessor directive ended directly before the token.
public unsigned int IsUnterminatedLiteral
Set to \c true if this token is an unterminated literal.
public unsigned int CanBreakBefore
\c true if it is allowed to break before this token.
public unsigned int ClosesTemplateDeclaration
\c true if this is the ">" of "template<..>".
public unsigned int StartsBinaryExpression
\c true if this token starts a binary expression, i.e. has at least one fake l_paren with a precedence greater than prec::Unknown.
public unsigned int EndsBinaryExpression
\c true if this token ends a binary expression.
public unsigned int PartOfMultiVariableDeclStmt
Only set if \c Type == \c TT_StartOfName.
public unsigned int ContinuesLineCommentSection
Only set to true if \c Type == \c TT_LineComment.
public unsigned int Finalized
If \c true, this token has been fully formatted (indented and potentially re-formatted inside), and we do not allow further formatting changes.
public unsigned int ClosesRequiresClause
\c true if this is the last token within requires clause.
private unsigned int BlockKind
Contains the kind of block if this token is a brace.
private unsigned int Decision
Stores the formatting decision for the token once it was made.
private unsigned int PackingKind
If this is an opening parenthesis, how are the parameters packed?
private unsigned int TypeIsFinalized
private clang::format::TokenType Type
public unsigned int NewlinesBefore = 0
This can be used to determine what the user wrote in the original code and thereby e.g. leave an empty line between two function definitions.
public unsigned int LastNewlineOffset = 0
The offset just past the last '\n' in this token's leading whitespace (relative to \c WhiteSpaceStart). 0 if there is no '\n'.
public unsigned int ColumnWidth = 0
The width of the non-whitespace parts of the token (or its first line for multi-line tokens) in columns. We need this to correctly measure number of columns a token spans.
public unsigned int LastLineColumnWidth = 0
Contains the width in columns of the last line of a multi-line token.
public unsigned int SpacesRequiredBefore = 0
The number of spaces that should be inserted before this token.
public unsigned int ParameterCount = 0
Number of parameters, if this is "(", "[" or "<".
public unsigned int BlockParameterCount = 0
Number of parameters that are nested blocks, if this is "(", "[" or "<".
public tok::TokenKind ParentBracket = tok::unknown
If this is a bracket ("<", "(", "[" or "{"), contains the kind of the surrounding bracket.
public unsigned int TotalLength = 0
The total length of the unwrapped line up to and including this token.
public unsigned int OriginalColumn = 0
The original 0-based column of this token, including expanded tabs. The configured TabWidth is used as tab width.
public unsigned int UnbreakableTailLength = 0
The length of following tokens until the next natural split point, or the next token that can be broken.
public unsigned int BindingStrength = 0
The binding strength of a token. This is a combined value of operator precedence, parenthesis nesting, etc.
public unsigned int NestingLevel = 0
The nesting level of this token, i.e. the number of surrounding (), [], {} or < >.
public unsigned int IndentLevel = 0
The indent level of this token. Copied from the surrounding line.
public unsigned int SplitPenalty = 0
Penalty for inserting a line break before this token.
public unsigned int LongestObjCSelectorName = 0
This being set to 0 means that the selectors should not be colon-aligned, e.g. because several of them are block-type.
public unsigned int ObjCSelectorNameParts = 0
If this is the first ObjC selector name in an ObjC method definition or call, this contains the number of parts that the whole selector consist of.
public unsigned int ParameterIndex = 0
The 0-based index of the parameter/argument. For ObjC it is set for the selector name token. For now calculated only for ObjC.
public SmallVector<prec::Level, 4> FakeLParens
If multiple fake parentheses start at a token, this vector stores them in reverse order, i.e. inner fake parenthesis first.
public unsigned int FakeRParens = 0
Insert this many fake ) after this token for correct indentation.
public unsigned int OperatorIndex = 0
If this is an operator (or "."/"->") in a sequence of operators with the same precedence, contains the 0-based operator index.
public clang::format::FormatToken* NextOperator = nullptr
If this is an operator (or "."/"->") in a sequence of operators with the same precedence, points to the next operator.
public clang::format::FormatToken* MatchingParen = nullptr
If this is a bracket, this points to the matching one.
public clang::format::FormatToken* Previous = nullptr
The previous token in the unwrapped line.
public clang::format::FormatToken* Next = nullptr
The next token in the unwrapped line.
public bool StartsColumn = false
The first token in set of column elements.
public bool ArrayInitializerLineStart = false
This notes the start of the line of an array initializer.
public bool IsArrayInitializer = false
This starts an array initializer.
public bool Optional = false
Is optional and can be removed.
public int8_t BraceCount = 0
Number of optional braces to be inserted after this token: -1: a single left brace 0: no braces >0: number of right braces
public SmallVector<clang::format::AnnotatedLine*, 1> Children
If this token starts a block, this contains all the unwrapped lines in it.
public llvm::Optional<MacroExpansion> MacroCtx
public bool MacroParent = false
When macro expansion introduces nodes with children, those are marked as\c MacroParent. FIXME: The formatting code currently hard-codes the assumption that child nodes are introduced by blocks following an opening brace. This is deeply baked into the code and disentangling this will require signficant refactorings. \c MacroParent allows us to special-case the cases in which we treat parents as block-openers for now.

Method Overview

Methods

FormatToken()

Declared at: clang/lib/Format/FormatToken.h:228

FormatToken(const clang::format::FormatToken&)

Declared at: clang/lib/Format/FormatToken.h:763

Parameters

const clang::format::FormatToken& 

bool canBePointerOrReferenceQualifier() const

Declared at: clang/lib/Format/FormatToken.h:572

bool closesBlockOrBlockTypeList(
    const clang::format::FormatStyle& Style) const

Description

Same as opensBlockOrBlockTypeList, but for the closing token.

Declared at: clang/lib/Format/FormatToken.h:738

Parameters

const clang::format::FormatStyle& Style

bool closesScope() const

Description

Returns whether \p Tok is )]} or a closing > of a template or in protos.

Declared at: clang/lib/Format/FormatToken.h:604

bool closesScopeAfterBlock() const

Declared at: clang/lib/Format/FormatToken.h:535

void copyFrom(
    const clang::format::FormatToken& Tok)

Declared at: clang/lib/Format/FormatToken.h:759

Parameters

const clang::format::FormatToken& Tok

template <typename A, typename... Ts>
bool endsSequence(A K1, Ts... Tokens) const

Description

\c true if this token ends a sequence with the given tokens in order, following the ``Previous`` pointers, ignoring comments. For example, given tokens [T1, T2, T3], the function returns true if 3 tokens ending at this (ignoring comments) are [T3, T2, T1]. In other words, the tokens passed to this function need to the reverse of the order the tokens appear in code.

Declared at: clang/lib/Format/FormatToken.h:557

Templates

A
Ts

Parameters

A K1
Ts... Tokens

template <typename A, typename... Ts>
bool endsSequenceInternal(A K1,
                          Ts... Tokens) const

Declared at: clang/lib/Format/FormatToken.h:786

Templates

A
Ts

Parameters

A K1
Ts... Tokens

template <typename A, typename... Ts>
bool endsSequenceInternal(A K1) const

Declared at: clang/lib/Format/FormatToken.h:779

Templates

A
Ts

Parameters

A K1

clang::format::BraceBlockKind getBlockKind() const

Declared at: clang/lib/Format/FormatToken.h:310

clang::format::FormatDecision getDecision() const

Declared at: clang/lib/Format/FormatToken.h:323

const clang::format::FormatToken*
getNamespaceToken() const

Description

Return the actual namespace token, if this token starts a namespace block.

Declared at: clang/lib/Format/FormatToken.h:746

const clang::format::FormatToken*
getNextNonComment() const

Description

Returns the next token ignoring comments.

Declared at: clang/lib/Format/FormatToken.h:713

clang::format::ParameterPackingKind
getPackingKind() const

Declared at: clang/lib/Format/FormatToken.h:336

prec::Level getPrecedence() const

Declared at: clang/lib/Format/FormatToken.h:699

clang::format::FormatToken*
getPreviousNonComment() const

Description

Returns the previous token ignoring comments.

Declared at: clang/lib/Format/FormatToken.h:705

clang::SourceLocation getStartOfNonWhitespace()
    const

Description

Returns actual token start location without leading escaped newlines and whitespace. This can be different to Tok.getLocation(), which includes leading escaped newlines.

Declared at: clang/lib/Format/FormatToken.h:689

clang::format::TokenType getType() const

Description

Returns the token's type, e.g. whether "<" is a template opener or binary operator.

Declared at: clang/lib/Format/FormatToken.h:351

bool hasWhitespaceBefore() const

Description

Returns \c true if the range of whitespace immediately preceding the \c Token is not empty.

Declared at: clang/lib/Format/FormatToken.h:695

bool is(tok::TokenKind Kind) const

Declared at: clang/lib/Format/FormatToken.h:509

Parameters

tok::TokenKind Kind

bool is(
    clang::format::ParameterPackingKind PPK) const

Declared at: clang/lib/Format/FormatToken.h:519

Parameters

clang::format::ParameterPackingKind PPK

bool is(clang::format::BraceBlockKind BBK) const

Declared at: clang/lib/Format/FormatToken.h:518

Parameters

clang::format::BraceBlockKind BBK

bool is(tok::PPKeywordKind Kind) const

Declared at: clang/lib/Format/FormatToken.h:514

Parameters

tok::PPKeywordKind Kind

bool is(const clang::IdentifierInfo* II) const

Declared at: clang/lib/Format/FormatToken.h:511

Parameters

const clang::IdentifierInfo* II

bool is(clang::format::TokenType TT) const

Declared at: clang/lib/Format/FormatToken.h:510

Parameters

clang::format::TokenType TT

bool isAccessSpecifier(
    bool ColonRequired = true) const

Declared at: clang/lib/Format/FormatToken.h:567

Parameters

bool ColonRequired = true

bool isBinaryOperator() const

Declared at: clang/lib/Format/FormatToken.h:636

bool isCppStructuredBinding(
    const clang::format::FormatStyle& Style) const

Description

Returns whether the token is the left square bracket of a C++ structured binding declaration.

Declared at: clang/lib/Format/FormatToken.h:726

Parameters

const clang::format::FormatStyle& Style

bool isFunctionLikeKeyword() const

Description

Returns \c true if this is a keyword that can be used like a function call (e.g. sizeof, typeid, ...).

Declared at: clang/lib/Format/FormatToken.h:648

bool isIf(bool AllowConstexprMacro = true) const

Declared at: clang/lib/Format/FormatToken.h:530

Parameters

bool AllowConstexprMacro = true

bool isLabelString() const

Description

Returns \c true if this is a string literal that's like a label, e.g. ends with "=" or ":".

Declared at: clang/lib/Format/FormatToken.h:671

bool isMemberAccess() const

Description

Returns \c true if this is a "." or "->" accessing a member.

Declared at: clang/lib/Format/FormatToken.h:614

template <typename T>
bool isNot(T Kind) const

Declared at: clang/lib/Format/FormatToken.h:528

Templates

T

Parameters

T Kind

bool isObjCAccessSpecifier() const

Declared at: clang/lib/Format/FormatToken.h:584

bool isObjCAtKeyword(
    tok::ObjCKeywordKind Kind) const

Declared at: clang/lib/Format/FormatToken.h:563

Parameters

tok::ObjCKeywordKind Kind

template <typename A, typename B>
bool isOneOf(A K1, B K2) const

Declared at: clang/lib/Format/FormatToken.h:521

Templates

A
B

Parameters

A K1
B K2

template <typename A, typename B, typename... Ts>
bool isOneOf(A K1, B K2, Ts... Ks) const

Declared at: clang/lib/Format/FormatToken.h:525

Templates

A
B
Ts

Parameters

A K1
B K2
Ts... Ks

bool isSimpleTypeSpecifier() const

Description

Determine whether the token is a simple-type-specifier.

Declared at: clang/lib/Format/FormatToken.h:580

bool isStringLiteral() const

Declared at: clang/lib/Format/FormatToken.h:561

bool isTrailingComment() const

Declared at: clang/lib/Format/FormatToken.h:641

bool isTypeFinalized() const

Declared at: clang/lib/Format/FormatToken.h:369

bool isTypeOrIdentifier() const

Declared at: clang/lib/Format/FormatToken.h:582

bool isUnaryOperator() const

Declared at: clang/lib/Format/FormatToken.h:620

bool opensBlockOrBlockTypeList(
    const clang::format::FormatStyle& Style) const

Description

Returns \c true if this tokens starts a block-type list, i.e. a list that should be indented with a block indent.

Declared at: clang/lib/Format/FormatToken.h:722

Parameters

const clang::format::FormatStyle& Style

bool opensScope() const

Description

Returns whether \p Tok is ([{ or an opening < of a template or in protos.

Declared at: clang/lib/Format/FormatToken.h:594

void overwriteFixedType(
    clang::format::TokenType T)

Declared at: clang/lib/Format/FormatToken.h:365

Parameters

clang::format::TokenType T

void setBlockKind(
    clang::format::BraceBlockKind BBK)

Declared at: clang/lib/Format/FormatToken.h:313

Parameters

clang::format::BraceBlockKind BBK

void setDecision(clang::format::FormatDecision D)

Declared at: clang/lib/Format/FormatToken.h:326

Parameters

clang::format::FormatDecision D

void setFinalizedType(clang::format::TokenType T)

Description

Sets the type and also the finalized flag. This prevents the type to be reset in TokenAnnotator::resetTokenMetadata(). If the type needs to be set to another one please use overwriteFixedType, or even better remove the need to reassign the type.

Declared at: clang/lib/Format/FormatToken.h:361

Parameters

clang::format::TokenType T

void setPackingKind(
    clang::format::ParameterPackingKind K)

Declared at: clang/lib/Format/FormatToken.h:339

Parameters

clang::format::ParameterPackingKind K

void setType(clang::format::TokenType T)

Declared at: clang/lib/Format/FormatToken.h:352

Parameters

clang::format::TokenType T

template <typename A, typename... Ts>
bool startsSequence(A K1, Ts... Tokens) const

Description

\c true if this token starts a sequence with the given tokens in order, following the ``Next`` pointers, ignoring comments.

Declared at: clang/lib/Format/FormatToken.h:546

Templates

A
Ts

Parameters

A K1
Ts... Tokens

template <typename A, typename... Ts>
bool startsSequenceInternal(A K1,
                            Ts... Tokens) const

Declared at: clang/lib/Format/FormatToken.h:767

Templates

A
Ts

Parameters

A K1
Ts... Tokens

template <typename A>
bool startsSequenceInternal(A K1) const

Declared at: clang/lib/Format/FormatToken.h:773

Templates

A

Parameters

A K1