The string tokenizer class allows an application to break a string into tokens The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis. More...

#include <StringTokenizer.hpp>

Public Member Functions
	StringTokenizer (const std::string &str, const std::string &delim=" ;\t\n\r")
	Constructs a string tokenizer for the specified string.

	StringTokenizer (std::string::const_iterator &first, std::string::const_iterator &last, const std::string &delim=" ;\t\n\r")
	Constructs a string tokenizer for an iterator range [first, last).

	StringTokenizer (const std::string &str, const std::string &delim, bool returnTokens)
	Constructs a string tokenizer for the specified string.

int	countTokens ()
	Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.

bool	hasMoreTokens ()
	Tests if there are more tokens available from this tokenizer's string.

std::string	nextToken ()
	Returns the next token from this string tokenizer.

void	skipToken ()
	Skips the next token from this string tokenizer.

bool	nextTokenAsBool ()
	Returns the next token from this string tokenizer as a bool.

int	nextTokenAsInt ()
	Returns the next token from this string tokenizer as an integer.

float	nextTokenAsFloat ()
	Returns the next token from this string tokenizer as a float.

RealType	nextTokenAsDouble ()
	Returns the next token from this string tokenizer as a RealType.

std::string	peekNextToken ()
	Returns the next token without advancing the position of the StringTokenizer.

const std::string &	getDelimiters ()
	Returns the current delimiter set of this string tokenizer.

const std::string &	getOriginal ()
	Returns the original string before tokenizing.

std::vector< std::string >	getAllTokens ()
	Returns all of the tokens.

std::string	getRemainingString ()
	Returns the remaining unparsed string.

Detailed Description

The string tokenizer class allows an application to break a string into tokens The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of two ways, depending on whether it was created with the returnTokens flag having the value true or false.

Definition at line 71 of file StringTokenizer.hpp.

Constructor & Destructor Documentation

◆ StringTokenizer() [1/3]

OpenMD::StringTokenizer::StringTokenizer	(	const std::string &	str,
		const std::string &	delim = " ;\t\n\r" )

Constructs a string tokenizer for the specified string.

The characters in the delim argument are the delimiters for separating tokens. characters are skipped and only serve as separators between tokens.

Parameters

str	a string to be parsed.
delim	the delimiters, default value is " ;\t\n\r".

Note: this is still a little bit java like implementation. Pure c++ one should use TokenIterator. Boost's tokenizer class is one of them

Definition at line 53 of file StringTokenizer.cpp.

◆ StringTokenizer() [2/3]

OpenMD::StringTokenizer::StringTokenizer	(	std::string::const_iterator &	first,
		std::string::const_iterator &	last,
		const std::string &	delim = " ;\t\n\r" )

Constructs a string tokenizer for an iterator range [first, last).

The characters in the delim argument are the delimiters for separating tokens. characters are skipped and only serve as separators between tokens.

Parameters

first	begin iterator
last	end iterator
delim	the delimiters, default value is " ;\t\n\r".

Note: this is still a little bit java like implementation. Pure c++ one should use TokenIterator. Boost's tokenizer class is one of them

Definition at line 59 of file StringTokenizer.cpp.

◆ StringTokenizer() [3/3]

OpenMD::StringTokenizer::StringTokenizer	(	const std::string &	str,
		const std::string &	delim,
		bool	returnTokens )

Constructs a string tokenizer for the specified string.

The characters in the delim argument are the delimiters for separating tokens. If the returnTokens flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

Parameters

str	a string to be parsed.
delim	the delimiters.
returnTokens	flag indicating whether to return the delimiters as tokens.

Definition at line 66 of file StringTokenizer.cpp.

Member Function Documentation

◆ countTokens()

int OpenMD::StringTokenizer::countTokens ( )

Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.

Returns: the number of tokens remaining in the string using the current delimiter set.

Definition at line 77 of file StringTokenizer.cpp.

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ getAllTokens()

std::vector< std::string > OpenMD::StringTokenizer::getAllTokens ( )

Returns all of the tokens.

Returns: all of the tokens

Definition at line 220 of file StringTokenizer.cpp.

References hasMoreTokens(), and nextToken().

◆ getDelimiters()

const std::string & OpenMD::StringTokenizer::getDelimiters ( )

inline

Returns the current delimiter set of this string tokenizer.

Returns: the current delimiter set

Definition at line 184 of file StringTokenizer.hpp.

◆ getOriginal()

const std::string & OpenMD::StringTokenizer::getOriginal ( )

inline

Returns the original string before tokenizing.

Returns: the original string before tokenizing

Definition at line 190 of file StringTokenizer.hpp.

◆ getRemainingString()

std::string OpenMD::StringTokenizer::getRemainingString ( )

Returns the remaining unparsed string.

Returns: the remaining unparsed string

Definition at line 235 of file StringTokenizer.cpp.

◆ hasMoreTokens()

bool OpenMD::StringTokenizer::hasMoreTokens ( )

Tests if there are more tokens available from this tokenizer's string.

Returns: true if there are more tokens available from this tokenizer's string, false otherwise

Definition at line 105 of file StringTokenizer.cpp.

Referenced by OpenMD::SimCreator::createSim(), and getAllTokens().

◆ nextToken()

std::string OpenMD::StringTokenizer::nextToken ( )

Returns the next token from this string tokenizer.

Returns: the next token from this string tokenizer.

Exceptions

NoSuchElementException if there are no more tokens in this tokenizer's string

Definition at line 123 of file StringTokenizer.cpp.

Referenced by OpenMD::SimCreator::createSim(), getAllTokens(), nextTokenAsBool(), nextTokenAsDouble(), nextTokenAsFloat(), nextTokenAsInt(), and OpenMD::DumpReader::parseSiteLine().

◆ nextTokenAsBool()

bool OpenMD::StringTokenizer::nextTokenAsBool ( )

Returns the next token from this string tokenizer as a bool.

Returns: the next token from this string tokenizer as a bool.

Definition at line 163 of file StringTokenizer.cpp.

References nextToken().

◆ nextTokenAsDouble()

RealType OpenMD::StringTokenizer::nextTokenAsDouble ( )

Returns the next token from this string tokenizer as a RealType.

Returns: the next token from this string tokenizer as a RealType.

Definition at line 190 of file StringTokenizer.cpp.

References nextToken().

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ nextTokenAsFloat()

float OpenMD::StringTokenizer::nextTokenAsFloat ( )

Returns the next token from this string tokenizer as a float.

Returns: the next token from this string tokenizer as a float.

Definition at line 184 of file StringTokenizer.cpp.

References nextToken().

◆ nextTokenAsInt()

int OpenMD::StringTokenizer::nextTokenAsInt ( )

Returns the next token from this string tokenizer as an integer.

Returns: the next token from this string tokenizer as an integer.

Definition at line 178 of file StringTokenizer.cpp.

References nextToken().

Referenced by OpenMD::SimCreator::createSim(), and OpenMD::DumpReader::parseSiteLine().

◆ peekNextToken()

std::string OpenMD::StringTokenizer::peekNextToken ( )

Returns the next token without advancing the position of the StringTokenizer.

Returns: the next token

Definition at line 196 of file StringTokenizer.cpp.

Referenced by OpenMD::DumpReader::parseSiteLine().

◆ skipToken()

void OpenMD::StringTokenizer::skipToken ( )

Skips the next token from this string tokenizer.

Exceptions

NoSuchElementException if there are no more tokens in this tokenizer's string

Definition at line 146 of file StringTokenizer.cpp.

The documentation for this class was generated from the following files:

utils/StringTokenizer.hpp
utils/StringTokenizer.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ StringTokenizer() [1/3]

◆ StringTokenizer() [2/3]

◆ StringTokenizer() [3/3]

Member Function Documentation

◆ countTokens()

◆ getAllTokens()

◆ getDelimiters()

◆ getOriginal()

◆ getRemainingString()

◆ hasMoreTokens()

◆ nextToken()

◆ nextTokenAsBool()

◆ nextTokenAsDouble()

◆ nextTokenAsFloat()

◆ nextTokenAsInt()

◆ peekNextToken()

◆ skipToken()