The string tokenizer class allows an application to break a string into tokens The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis. More...

#include <StringTokenizer.hpp>

Public Member Functions
	StringTokenizer (const std::string &str, const std::string &delim=" ;\t\n\r")
	Constructs a string tokenizer for the specified string.
	StringTokenizer (std::string::const_iterator &first, std::string::const_iterator &last, const std::string &delim=" ;\t\n\r")
	Constructs a string tokenizer for an iterator range [first, last).
	StringTokenizer (const std::string &str, const std::string &delim, bool returnTokens)
	Constructs a string tokenizer for the specified string.
int	countTokens ()
	Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.
bool	hasMoreTokens ()
	Tests if there are more tokens available from this tokenizer's string.
std::string	nextToken ()
	Returns the next token from this string tokenizer.
void	skipToken ()
	Skips the next token from this string tokenizer.
bool	nextTokenAsBool ()
	Returns the next token from this string tokenizer as a bool.
int	nextTokenAsInt ()
	Returns the next token from this string tokenizer as an integer.
float	nextTokenAsFloat ()
	Returns the next token from this string tokenizer as a float.
RealType	nextTokenAsDouble ()
	Returns the next token from this string tokenizer as a RealType.
std::string	peekNextToken ()
	Returns the next token without advancing the position of the StringTokenizer.
const std::string &	getDelimiters ()
	Returns the current delimiter set of this string tokenizer.
const std::string &	getOriginal ()
	Returns the original string before tokenizing.
std::vector< std::string >	getAllTokens ()
	Returns all of the tokens.
std::string	getRemainingString () const
	Returns the remaining unparsed string.

Detailed Description

The string tokenizer class allows an application to break a string into tokens The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of two ways, depending on whether it was created with the returnTokens flag having the value true or false.

Definition at line 74 of file StringTokenizer.hpp.

Constructor & Destructor Documentation

◆ StringTokenizer() [1/3]

OpenMD::StringTokenizer::StringTokenizer	(	const std::string &	str,
		const std::string &	delim = " ;\t\n\r" )

Constructs a string tokenizer for the specified string.

The characters in the delim argument are the delimiters for separating tokens. characters are skipped and only serve as separators between tokens.

Parameters

str	a string to be parsed.
delim	the delimiters, default value is " ;\t\n\r".

Note: this is still a little bit java like implementation. Pure c++ one should use TokenIterator. Boost's tokenizer class is one of them

Definition at line 56 of file StringTokenizer.cpp.

◆ StringTokenizer() [2/3]

OpenMD::StringTokenizer::StringTokenizer	(	std::string::const_iterator &	first,
		std::string::const_iterator &	last,
		const std::string &	delim = " ;\t\n\r" )

Constructs a string tokenizer for an iterator range [first, last).

The characters in the delim argument are the delimiters for separating tokens. characters are skipped and only serve as separators between tokens.

Parameters

first	begin iterator
last	end iterator
delim	the delimiters, default value is " ;\t\n\r".

Note: this is still a little bit java like implementation. Pure c++ one should use TokenIterator. Boost's tokenizer class is one of them

Definition at line 62 of file StringTokenizer.cpp.

◆ StringTokenizer() [3/3]

OpenMD::StringTokenizer::StringTokenizer	(	const std::string &	str,
		const std::string &	delim,
		bool	returnTokens )

Constructs a string tokenizer for the specified string.

The characters in the delim argument are the delimiters for separating tokens. If the returnTokens flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

Parameters

str	a string to be parsed.
delim	the delimiters.
returnTokens	flag indicating whether to return the delimiters as tokens.

Definition at line 69 of file StringTokenizer.cpp.

Member Function Documentation

◆ countTokens()

int OpenMD::StringTokenizer::countTokens ( )

Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.

Returns: the number of tokens remaining in the string using the current delimiter set.

Definition at line 80 of file StringTokenizer.cpp.

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ getAllTokens()

std::vector< std::string > OpenMD::StringTokenizer::getAllTokens ( )

Returns all of the tokens.

Returns: all of the tokens

Definition at line 223 of file StringTokenizer.cpp.

References hasMoreTokens(), and nextToken().

◆ getDelimiters()

const std::string & OpenMD::StringTokenizer::getDelimiters ( )

inline

Returns the current delimiter set of this string tokenizer.

Returns: the current delimiter set

Definition at line 187 of file StringTokenizer.hpp.

◆ getOriginal()

const std::string & OpenMD::StringTokenizer::getOriginal ( )

inline

Returns the original string before tokenizing.

Returns: the original string before tokenizing

Definition at line 193 of file StringTokenizer.hpp.

◆ getRemainingString()

std::string OpenMD::StringTokenizer::getRemainingString ( ) const

Returns the remaining unparsed string.

Returns: the remaining unparsed string

Definition at line 238 of file StringTokenizer.cpp.

◆ hasMoreTokens()

bool OpenMD::StringTokenizer::hasMoreTokens ( )

Tests if there are more tokens available from this tokenizer's string.

Returns: true if there are more tokens available from this tokenizer's string, false otherwise

Definition at line 108 of file StringTokenizer.cpp.

Referenced by OpenMD::SimCreator::createSim(), and getAllTokens().

◆ nextToken()

std::string OpenMD::StringTokenizer::nextToken ( )

Returns the next token from this string tokenizer.

Returns: the next token from this string tokenizer.

Exceptions

NoSuchElementException if there are no more tokens in this tokenizer's string

Definition at line 126 of file StringTokenizer.cpp.

Referenced by OpenMD::SimCreator::createSim(), getAllTokens(), nextTokenAsBool(), nextTokenAsDouble(), nextTokenAsFloat(), nextTokenAsInt(), and OpenMD::DumpReader::parseSiteLine().

◆ nextTokenAsBool()

bool OpenMD::StringTokenizer::nextTokenAsBool ( )

Returns the next token from this string tokenizer as a bool.

Returns: the next token from this string tokenizer as a bool.

Definition at line 166 of file StringTokenizer.cpp.

References nextToken().

◆ nextTokenAsDouble()

RealType OpenMD::StringTokenizer::nextTokenAsDouble ( )

Returns the next token from this string tokenizer as a RealType.

Returns: the next token from this string tokenizer as a RealType.

Definition at line 193 of file StringTokenizer.cpp.

References nextToken().

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ nextTokenAsFloat()

float OpenMD::StringTokenizer::nextTokenAsFloat ( )

Returns the next token from this string tokenizer as a float.

Returns: the next token from this string tokenizer as a float.

Definition at line 187 of file StringTokenizer.cpp.

References nextToken().

◆ nextTokenAsInt()

int OpenMD::StringTokenizer::nextTokenAsInt ( )

Returns the next token from this string tokenizer as an integer.

Returns: the next token from this string tokenizer as an integer.

Definition at line 181 of file StringTokenizer.cpp.

References nextToken().

Referenced by OpenMD::SimCreator::createSim(), and OpenMD::DumpReader::parseSiteLine().

◆ peekNextToken()

std::string OpenMD::StringTokenizer::peekNextToken ( )

Returns the next token without advancing the position of the StringTokenizer.

Returns: the next token

Definition at line 199 of file StringTokenizer.cpp.

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ skipToken()

void OpenMD::StringTokenizer::skipToken ( )

Skips the next token from this string tokenizer.

Exceptions

NoSuchElementException if there are no more tokens in this tokenizer's string

Definition at line 149 of file StringTokenizer.cpp.

The documentation for this class was generated from the following files:

utils/StringTokenizer.hpp
utils/StringTokenizer.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ StringTokenizer() [1/3]

◆ StringTokenizer() [2/3]

◆ StringTokenizer() [3/3]

Member Function Documentation

◆ countTokens()

◆ getAllTokens()

◆ getDelimiters()

◆ getOriginal()

◆ getRemainingString()

◆ hasMoreTokens()

◆ nextToken()

◆ nextTokenAsBool()

◆ nextTokenAsDouble()

◆ nextTokenAsFloat()

◆ nextTokenAsInt()

◆ peekNextToken()

◆ skipToken()