class BreakableToken

Declaration

class BreakableToken { /* full declaration omitted */ };

Description

Base class for tokens / ranges of tokens that can allow breaking within the tokens - for example, to avoid whitespace beyond the column limit, or to reflow text. Generally, a breakable token consists of logical lines, addressed by a line index. For example, in a sequence of line comments, each line comment is its own logical line; similarly, for a block comment, each line in the block comment is on its own logical line. There are two methods to compute the layout of the token: - getRangeLength measures the number of columns needed for a range of text within a logical line, and - getContentStartColumn returns the start column at which we want the content of a logical line to start (potentially after introducing a line break). The mechanism to adapt the layout of the breakable token is organised around the concept of a \c Split, which is a whitespace range that signifies a position of the content of a token where a reformatting might be done. Operating with splits is divided into two operations: - getSplit, for finding a split starting at a position, - insertBreak, for executing the split using a whitespace manager. There is a pair of operations that are used to compress a long whitespace range with a single space if that will bring the line length under the column limit: - getLineLengthAfterCompression, for calculating the size in columns of the line after a whitespace range has been compressed, and - compressWhitespace, for executing the whitespace compression using a whitespace manager; note that the compressed whitespace may be in the middle of the original line and of the reformatted line. For tokens where the whitespace before each line needs to be also reformatted, for example for tokens supporting reflow, there are analogous operations that might be executed before the main line breaking occurs: - getReflowSplit, for finding a split such that the content preceding it needs to be specially reflown, - reflow, for executing the split using a whitespace manager, - introducesBreakBefore, for checking if reformatting the beginning of the content introduces a line break before it, - adaptStartOfLine, for executing the reflow using a whitespace manager. For tokens that require the whitespace after the last line to be reformatted, for example in multiline jsdoc comments that require the trailing '*/' to be on a line of itself, there are analogous operations that might be executed after the last line has been reformatted: - getSplitAfterLastLine, for finding a split after the last line that needs to be reflown, - replaceWhitespaceAfterLastLine, for executing the reflow using a whitespace manager.

Declared at: clang/lib/Format/BreakableToken.h:89

Member Variables

protected const clang::format::FormatToken& Tok
protected const bool InPPDirective
protected const encoding::Encoding Encoding
protected const clang::format::FormatStyle& Style

Method Overview

  • protected BreakableToken(const clang::format::FormatToken & Tok, bool InPPDirective, encoding::Encoding Encoding, const clang::format::FormatStyle & Style)
  • public virtual void adaptStartOfLine(unsigned int LineIndex, clang::format::WhitespaceManager & Whitespaces) const
  • public virtual void compressWhitespace(unsigned int LineIndex, unsigned int TailOffset, clang::format::BreakableToken::Split Split, clang::format::WhitespaceManager & Whitespaces) const
  • public virtual unsigned int getContentIndent(unsigned int LineIndex) const
  • public virtual unsigned int getContentStartColumn(unsigned int LineIndex, bool Break) const
  • public unsigned int getLengthAfterCompression(unsigned int RemainingTokenColumns, clang::format::BreakableToken::Split Split) const
  • public virtual unsigned int getLineCount() const
  • public virtual unsigned int getRangeLength(unsigned int LineIndex, unsigned int Offset, StringRef::size_type Length, unsigned int StartColumn) const
  • public virtual clang::format::BreakableToken::Split getReflowSplit(unsigned int LineIndex, const llvm::Regex & CommentPragmasRegex) const
  • public virtual unsigned int getRemainingLength(unsigned int LineIndex, unsigned int Offset, unsigned int StartColumn) const
  • public virtual clang::format::BreakableToken::Split getSplit(unsigned int LineIndex, unsigned int TailOffset, unsigned int ColumnLimit, unsigned int ContentStartColumn, const llvm::Regex & CommentPragmasRegex) const
  • public virtual clang::format::BreakableToken::Split getSplitAfterLastLine(unsigned int TailOffset) const
  • public virtual void insertBreak(unsigned int LineIndex, unsigned int TailOffset, clang::format::BreakableToken::Split Split, unsigned int ContentIndent, clang::format::WhitespaceManager & Whitespaces) const
  • public virtual bool introducesBreakBeforeToken() const
  • public virtual void reflow(unsigned int LineIndex, clang::format::WhitespaceManager & Whitespaces) const
  • public void replaceWhitespaceAfterLastLine(unsigned int TailOffset, clang::format::BreakableToken::Split SplitAfterLastLine, clang::format::WhitespaceManager & Whitespaces) const
  • public virtual bool supportsReflow() const
  • public virtual void updateNextToken(clang::format::LineState & State) const
  • public virtual ~BreakableToken()

Methods

BreakableToken(
    const clang::format::FormatToken& Tok,
    bool InPPDirective,
    encoding::Encoding Encoding,
    const clang::format::FormatStyle& Style)

Declared at: clang/lib/Format/BreakableToken.h:234

Parameters

const clang::format::FormatToken& Tok
bool InPPDirective
encoding::Encoding Encoding
const clang::format::FormatStyle& Style

virtual void adaptStartOfLine(
    unsigned int LineIndex,
    clang::format::WhitespaceManager& Whitespaces)
    const

Description

Replaces the whitespace between \p LineIndex-1 and \p LineIndex.

Declared at: clang/lib/Format/BreakableToken.h:206

Parameters

unsigned int LineIndex
clang::format::WhitespaceManager& Whitespaces

virtual void compressWhitespace(
    unsigned int LineIndex,
    unsigned int TailOffset,
    clang::format::BreakableToken::Split Split,
    clang::format::WhitespaceManager& Whitespaces)
    const

Description

Replaces the whitespace range described by \p Split with a single space.

Declared at: clang/lib/Format/BreakableToken.h:174

Parameters

unsigned int LineIndex
unsigned int TailOffset
clang::format::BreakableToken::Split Split
clang::format::WhitespaceManager& Whitespaces

virtual unsigned int getContentIndent(
    unsigned int LineIndex) const

Declared at: clang/lib/Format/BreakableToken.h:149

Parameters

unsigned int LineIndex

virtual unsigned int getContentStartColumn(
    unsigned int LineIndex,
    bool Break) const

Description

Returns the column at which content in line \p LineIndex starts, assuming no reflow. If \p Break is true, returns the column at which the line should start after the line break. If \p Break is false, returns the column at which the line itself will start.

Declared at: clang/lib/Format/BreakableToken.h:135

Parameters

unsigned int LineIndex
bool Break

unsigned int getLengthAfterCompression(
    unsigned int RemainingTokenColumns,
    clang::format::BreakableToken::Split Split)
    const

Description

Returns the number of columns needed to format\p RemainingTokenColumns, assuming that Split is within the range measured by \p RemainingTokenColumns, and that the whitespace in Split is reduced to a single space.

Declared at: clang/lib/Format/BreakableToken.h:169

Parameters

unsigned int RemainingTokenColumns
clang::format::BreakableToken::Split Split

virtual unsigned int getLineCount() const

Description

Returns the number of lines in this token in the original code.

Declared at: clang/lib/Format/BreakableToken.h:97

virtual unsigned int getRangeLength(
    unsigned int LineIndex,
    unsigned int Offset,
    StringRef::size_type Length,
    unsigned int StartColumn) const

Description

Returns the number of columns required to format the text in the byte range [\p Offset, \p Offset \c + \p Length). \p Offset is the byte offset from the start of the content of the line at \p LineIndex. \p StartColumn is the column at which the text starts in the formatted file, needed to compute tab stops correctly.

Declared at: clang/lib/Format/BreakableToken.h:107

Parameters

unsigned int LineIndex
unsigned int Offset
StringRef::size_type Length
unsigned int StartColumn

virtual clang::format::BreakableToken::Split
getReflowSplit(
    unsigned int LineIndex,
    const llvm::Regex& CommentPragmasRegex) const

Description

Returns a whitespace range (offset, length) of the content at \p LineIndex such that the content of that line is reflown to the end of the previous one. Returning (StringRef::npos, 0) indicates reflowing is not possible. The range will include any whitespace preceding the specified line's content. If the split is not contained within one token, for example when reflowing line comments, returns (0, <length >).

Declared at: clang/lib/Format/BreakableToken.h:192

Parameters

unsigned int LineIndex
const llvm::Regex& CommentPragmasRegex

virtual unsigned int getRemainingLength(
    unsigned int LineIndex,
    unsigned int Offset,
    unsigned int StartColumn) const

Description

Returns the number of columns required to format the text following the byte \p Offset in the line \p LineIndex, including potentially unbreakable sequences of tokens following after the end of the token. \p Offset is the byte offset from the start of the content of the line at \p LineIndex. \p StartColumn is the column at which the text starts in the formatted file, needed to compute tab stops correctly. For breakable tokens that never use extra space at the end of a line, this is equivalent to getRangeLength with a Length of StringRef::npos.

Declared at: clang/lib/Format/BreakableToken.h:123

Parameters

unsigned int LineIndex
unsigned int Offset
unsigned int StartColumn

virtual clang::format::BreakableToken::Split
getSplit(
    unsigned int LineIndex,
    unsigned int TailOffset,
    unsigned int ColumnLimit,
    unsigned int ContentStartColumn,
    const llvm::Regex& CommentPragmasRegex) const

Description

Returns a range (offset, length) at which to break the line at\p LineIndex, if previously broken at \p TailOffset. If possible, do not violate \p ColumnLimit, assuming the text starting at \p TailOffset in the token is formatted starting at ContentStartColumn in the reformatted file.

Declared at: clang/lib/Format/BreakableToken.h:156

Parameters

unsigned int LineIndex
unsigned int TailOffset
unsigned int ColumnLimit
unsigned int ContentStartColumn
const llvm::Regex& CommentPragmasRegex

virtual clang::format::BreakableToken::Split
getSplitAfterLastLine(
    unsigned int TailOffset) const

Description

Returns a whitespace range (offset, length) of the content at the last line that needs to be reformatted after the last line has been reformatted. A result having offset == StringRef::npos means that no reformat is necessary.

Declared at: clang/lib/Format/BreakableToken.h:215

Parameters

unsigned int TailOffset

virtual void insertBreak(
    unsigned int LineIndex,
    unsigned int TailOffset,
    clang::format::BreakableToken::Split Split,
    unsigned int ContentIndent,
    clang::format::WhitespaceManager& Whitespaces)
    const

Description

Emits the previously retrieved \p Split via \p Whitespaces.

Declared at: clang/lib/Format/BreakableToken.h:161

Parameters

unsigned int LineIndex
unsigned int TailOffset
clang::format::BreakableToken::Split Split
unsigned int ContentIndent
clang::format::WhitespaceManager& Whitespaces

virtual bool introducesBreakBeforeToken() const

Description

Returns whether there will be a line break at the start of the token.

Declared at: clang/lib/Format/BreakableToken.h:203

virtual void reflow(
    unsigned int LineIndex,
    clang::format::WhitespaceManager& Whitespaces)
    const

Description

Reflows the current line into the end of the previous one.

Declared at: clang/lib/Format/BreakableToken.h:198

Parameters

unsigned int LineIndex
clang::format::WhitespaceManager& Whitespaces

void replaceWhitespaceAfterLastLine(
    unsigned int TailOffset,
    clang::format::BreakableToken::Split
        SplitAfterLastLine,
    clang::format::WhitespaceManager& Whitespaces)
    const

Description

Replaces the whitespace from \p SplitAfterLastLine on the last line after the last line has been formatted by performing a reformatting.

Declared at: clang/lib/Format/BreakableToken.h:221

Parameters

unsigned int TailOffset
clang::format::BreakableToken::Split SplitAfterLastLine
clang::format::WhitespaceManager& Whitespaces

virtual bool supportsReflow() const

Description

Returns whether the token supports reflowing text.

Declared at: clang/lib/Format/BreakableToken.h:179

virtual void updateNextToken(
    clang::format::LineState& State) const

Description

Updates the next token of \p State to the next token after this one. This can be used when this token manages a set of underlying tokens as a unit and is responsible for the formatting of the them.

Declared at: clang/lib/Format/BreakableToken.h:231

Parameters

clang::format::LineState& State

virtual ~BreakableToken()

Declared at: clang/lib/Format/BreakableToken.h:94