com.fasterxml.jackson.core.sym

Class CharsToNameCanonicalizer

java.lang.Object

com.fasterxml.jackson.core.sym.CharsToNameCanonicalizer

public final class CharsToNameCanonicalizer
extends Object

This class is a kind of specialized type-safe Map, from char array to String value. Specialization means that in addition to type-safety and specific access patterns (key char array, Value optionally interned String; values added on access if necessary), and that instances are meant to be used concurrently, but by using well-defined mechanisms to obtain such concurrently usable instances. Main use for the class is to store symbol table information for things like compilers and parsers; especially when number of symbols (keywords) is limited.

For optimal performance, usage pattern should be one where matches should be very common (especially after "warm-up"), and as with most hash-based maps/sets, that hash codes are uniformly distributed. Also, collisions are slightly more expensive than with HashMap or HashSet, since hash codes are not used in resolving collisions; that is, equals() comparison is done with all symbols in same bucket index.
Finally, rehashing is also more expensive, as hash codes are not stored; rehashing requires all entries' hash codes to be recalculated. Reason for not storing hash codes is reduced memory usage, hoping for better memory locality.

Usual usage pattern is to create a single "master" instance, and either use that instance in sequential fashion, or to create derived "child" instances, which after use, are asked to return possible symbol additions to master instance. In either case benefit is that symbol table gets initialized so that further uses are more efficient, as eventually all symbols needed will already be in symbol table. At that point no more Symbol String allocations are needed, nor changes to symbol table itself.

Note that while individual SymbolTable instances are NOT thread-safe (much like generic collection classes), concurrently used "child" instances can be freely used without synchronization. However, using master table concurrently with child instances can only be done if access to master instance is read-only (i.e. no modifications done).

Field Summary

Fields

Modifier and Type	Field and Description
protected com.fasterxml.jackson.core.sym.CharsToNameCanonicalizer.Bucket[]	_buckets Overflow buckets; if primary doesn't match, lookup is done from here.
protected boolean	_canonicalize Whether any canonicalization should be attempted (whether using intern or not)
protected boolean	_dirty Flag that indicates if any changes have been made to the data; used to both determine if bucket array needs to be copied when (first) change is made, and potentially if updated bucket list is to be resync'ed back to master instance.
protected int	_flags
protected int	_indexMask Mask used to get index from hash values; equal to _buckets.length - 1, when _buckets.length is a power of two.
protected int	_longestCollisionList We need to keep track of the longest collision list; this is needed both to indicate problems with attacks and to allow flushing for other cases.
protected BitSet	_overflows Lazily constructed structure that is used to keep track of collision buckets that have overflowed once: this is used to detect likely attempts at denial-of-service attacks that uses hash collisions.
protected CharsToNameCanonicalizer	_parent Sharing of learnt symbols is done by optional linking of symbol table instances with their parents.
protected int	_size Current size (number of entries); needed to know if and when rehash.
protected int	_sizeThreshold Limit that indicates maximum size this instance can hold before it needs to be expanded and rehashed.
protected String[]	_symbols Primary matching symbols; it's expected most match occur from here.
protected static int	DEFAULT_T_SIZE Default initial table size.
static int	HASH_MULT
protected static int	MAX_T_SIZE Let's not expand symbol tables past some maximum size; this should protected against OOMEs caused by large documents with unique (~= random) names.

Method Summary

Methods

Modifier and Type	Method and Description
int	_hashToIndex(int rawHash) Helper method that takes in a "raw" hash value, shuffles it as necessary, and truncates to be used as the index.
int	bucketCount() Method for checking number of primary hash buckets this symbol table uses.
int	calcHash(char[] buffer, int start, int len) Implementation of a hashing method for variable length Strings.
int	calcHash(String key)
int	collisionCount() Method mostly needed by unit tests; calculates number of entries that are in collision list.
static CharsToNameCanonicalizer	createRoot() Method called to create root canonicalizer for a JsonFactory instance.
protected static CharsToNameCanonicalizer	createRoot(int hashSeed)
String	findSymbol(char[] buffer, int start, int len, int h)
int	hashSeed()
CharsToNameCanonicalizer	makeChild(int flags) "Factory" method; will create a new child instance of this symbol table.
int	maxCollisionLength() Method mostly needed by unit tests; calculates length of the longest collision chain.
boolean	maybeDirty()
void	release()
protected void	reportTooManyCollisions(int maxLen)
int	size()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HASH_MULT

public static final int HASH_MULT

See Also:

Constant Field Values

DEFAULT_T_SIZE

protected static final int DEFAULT_T_SIZE

Default initial table size. Shouldn't be miniscule (as there's cost to both array realloc and rehashing), but let's keep it reasonably small. For systems that properly reuse factories it doesn't matter either way; but when recreating factories often, initial overhead may dominate.

See Also:

Constant Field Values

MAX_T_SIZE

protected static final int MAX_T_SIZE

Let's not expand symbol tables past some maximum size; this should protected against OOMEs caused by large documents with unique (~= random) names.

See Also:

Constant Field Values

_parent

protected CharsToNameCanonicalizer _parent

Sharing of learnt symbols is done by optional linking of symbol table instances with their parents. When parent linkage is defined, and child instance is released (call to release), parent's shared tables may be updated from the child instance.

_flags

protected final int _flags

_canonicalize

protected boolean _canonicalize

Whether any canonicalization should be attempted (whether using intern or not)

_symbols

protected String[] _symbols

Primary matching symbols; it's expected most match occur from here.

_buckets

protected com.fasterxml.jackson.core.sym.CharsToNameCanonicalizer.Bucket[] _buckets

Overflow buckets; if primary doesn't match, lookup is done from here.

Note: Number of buckets is half of number of symbol entries, on assumption there's less need for buckets.

_size

protected int _size

Current size (number of entries); needed to know if and when rehash.

_sizeThreshold

protected int _sizeThreshold

Limit that indicates maximum size this instance can hold before it needs to be expanded and rehashed. Calculated using fill factor passed in to constructor.

_indexMask

protected int _indexMask

Mask used to get index from hash values; equal to _buckets.length - 1, when _buckets.length is a power of two.

_longestCollisionList

protected int _longestCollisionList

We need to keep track of the longest collision list; this is needed both to indicate problems with attacks and to allow flushing for other cases.

Since:

2.1

_dirty

protected boolean _dirty

Flag that indicates if any changes have been made to the data; used to both determine if bucket array needs to be copied when (first) change is made, and potentially if updated bucket list is to be resync'ed back to master instance.

_overflows

protected BitSet _overflows

Lazily constructed structure that is used to keep track of collision buckets that have overflowed once: this is used to detect likely attempts at denial-of-service attacks that uses hash collisions.

Since:

2.4

Method Detail

createRoot

public static CharsToNameCanonicalizer createRoot()

Method called to create root canonicalizer for a JsonFactory instance. Root instance is never used directly; its main use is for storing and sharing underlying symbol arrays as needed.

createRoot

protected static CharsToNameCanonicalizer createRoot(int hashSeed)

makeChild

public CharsToNameCanonicalizer makeChild(int flags)

"Factory" method; will create a new child instance of this symbol table. It will be a copy-on-write instance, ie. it will only use read-only copy of parent's data, but when changes are needed, a copy will be created.

Note: while this method is synchronized, it is generally not safe to both use makeChild/mergeChild, AND to use instance actively. Instead, a separate 'root' instance should be used on which only makeChild/mergeChild are called, but instance itself is not used as a symbol table.

release

public void release()

size

public int size()

bucketCount

public int bucketCount()

Method for checking number of primary hash buckets this symbol table uses.

Since:

2.1

maybeDirty

public boolean maybeDirty()

hashSeed

public int hashSeed()

collisionCount

public int collisionCount()

Method mostly needed by unit tests; calculates number of entries that are in collision list. Value can be at most (size() - 1), but should usually be much lower, ideally 0.

Since:

2.1

maxCollisionLength

public int maxCollisionLength()

Method mostly needed by unit tests; calculates length of the longest collision chain. This should typically be a low number, but may be up to size() - 1 in the pathological case

Since:

2.1

findSymbol

public String findSymbol(char[] buffer,
                int start,
                int len,
                int h)

_hashToIndex

public int _hashToIndex(int rawHash)

Helper method that takes in a "raw" hash value, shuffles it as necessary, and truncates to be used as the index.

calcHash

public int calcHash(char[] buffer,
           int start,
           int len)

Implementation of a hashing method for variable length Strings. Most of the time intention is that this calculation is done by caller during parsing, not here; however, sometimes it needs to be done for parsed "String" too.

Parameters:

len - Length of String; has to be at least 1 (caller guarantees this pre-condition)

calcHash

public int calcHash(String key)

reportTooManyCollisions

protected void reportTooManyCollisions(int maxLen)

Since:

2.1