class Token

Declaration

class Token { /* full declaration omitted */ };

Description

A token coming directly from a file or from a macro invocation. Has just enough information to locate the token in the source code. Can represent both expanded and spelled tokens.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:104

Member Variables

private clang::SourceLocation Location
private unsigned int Length
private tok::TokenKind Kind

Method Overview

  • public Token(clang::SourceLocation Location, unsigned int Length, tok::TokenKind Kind)
  • public Token(const clang::Token & T)
  • public std::string dumpForTests(const clang::SourceManager & SM) const
  • public clang::SourceLocation endLocation() const
  • public tok::TokenKind kind() const
  • public unsigned int length() const
  • public clang::SourceLocation location() const
  • public clang::syntax::FileRange range(const clang::SourceManager & SM) const
  • public static clang::syntax::FileRange range(const clang::SourceManager & SM, const syntax::Token & First, const syntax::Token & Last)
  • public std::string str() const
  • public llvm::StringRef text(const clang::SourceManager & SM) const

Methods

Token(clang::SourceLocation Location,
      unsigned int Length,
      tok::TokenKind Kind)

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:106

Parameters

clang::SourceLocation Location
unsigned int Length
tok::TokenKind Kind

Token(const clang::Token& T)

Description

EXPECTS: clang::Token is not an annotation token.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:108

Parameters

const clang::Token& T

std::string dumpForTests(
    const clang::SourceManager& SM) const

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:137

Parameters

const clang::SourceManager& SM

clang::SourceLocation endLocation() const

Description

Location right after the last character of a token.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:114

tok::TokenKind kind() const

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:110

unsigned int length() const

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:117

clang::SourceLocation location() const

Description

Location of the first character of a token.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:112

clang::syntax::FileRange range(
    const clang::SourceManager& SM) const

Description

Gets a range of this token. EXPECTS: token comes from a file, not from a macro expansion.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:128

Parameters

const clang::SourceManager& SM

static clang::syntax::FileRange range(
    const clang::SourceManager& SM,
    const syntax::Token& First,
    const syntax::Token& Last)

Description

Given two tokens inside the same file, returns a file range that starts at\p First and ends at \p Last. EXPECTS: First and Last are file tokens from the same file, Last starts after First.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:134

Parameters

const clang::SourceManager& SM
const syntax::Token& First
const syntax::Token& Last

std::string str() const

Description

For debugging purposes.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:139

llvm::StringRef text(
    const clang::SourceManager& SM) const

Description

Get the substring covered by the token. Note that will include all digraphs, newline continuations, etc. E.g. tokens for 'int' and in \ /// t both have the same kind tok::kw_int, but results of text() are different.

Declared at: clang/include/clang/Tooling/Syntax/Tokens.h:124

Parameters

const clang::SourceManager& SM