Flows are preallocated at startup and stored in protocol specific caches.
FlowKey is used for quick look up in the cache hash table.

Each flow may have associated inspectors:

* clouseau is the Wizard bound to the flow to help determine the
  appropriate service inspector

* gadget is the service inspector

* data is a passive service inspector such as a client config.

FlowData is used by various inspectors to store specific data on the flow
for later use.  Any inspector may store data on the flow, not just clouseau
gadget.

FlowData reference counts the associated inspector so that the inspector
can be freed (via garbage collection) after a reload.

There are many flags that may be set on a flow to indicate session tracking
state, disposition, etc.

==== High Availability

HighAvailability (ha.cc, ha.h) serves to synchronize session state between high
availability partners.  The primary purpose is to exchange session state with the
goal of keeping the session caches in sync.  Other clients may also register
with the HA service to exchange additional session data by implementing one or
more FlowHAClient classes.

HA uses Side Channel Connectors and/or DAQ module IOCTLs to transmit and receive
HA state messages.  A full duplex Side Channel is required for Side Channel
communications.  A DAQ module that supports the IOCTLs for setting and getting
HA state is required for a functional DAQ-backed setup.  Modules that do not
support these IOCTLs will silently fail and present no HA state on future
packets in the flow.  HA state will be queried from the DAQ module prior to new
flow creation when the appropriate DAQ packet message flag has been set on the
initiating packet.

The HA subsystem exchanges two high level message types:
  - DELETE: Indicate to the partner that a session has been removed.  No
additional HA client status will be exchanged.  (Not used for DAQ-backed
storage.)
  - UPDATE: Indicate all other state changes.  The message always includes
the session state and optionally may include state from other HA clients.
By default, the update messages are incremental.  In the case of DAQ-backed
storage, the update messages are always fully formed.

The HA subsystem implements these classes:
  - HighAvailabilityManager - A collection of static elements providing the
    top-most interface to HA capabilities.
  - HAMessage - A wrapper around the actual message and includes a cursor and
    convenience functions for producer/consumer activity.  Passed around
    among all message handing classes/methods.
  - HighAvailability - If HA is enabled, instantiated in each packet thread and
    provides all primary HA functionality for the thread.  Referenced via a
    THREAD_LOCAL object pointer.
  - FlowHAState - One per flow and referenced via a pointer in the Flow object.
    Contains all dynamic state information.  A set of get'ers and set'ers are
    implemented to help manage the flags, client pending flags, and timing
    information.
  - FlowHAClient - All HA coordination is managed via a set of FlowHAClient's.
    Every function that desires to use HA capabilities defines one FlowHAClient
    object within the packet processing thread.  Each client is provided a
    handle bit-mask as part of the pending logic.  The client may set the
    pending bit to indicate the desire to add content to the HA message.
    A maximum of 17 clients may existing (numbers 1-16, corresponding to the
    16 bits of the pending flags.  Client 0 is the 'session', is always present,
    and is handled as a special case.  Client 0 is the fundamental session HA
    state sync functionality.  Other clients are optional.


09/25/2023
In response to the need for more nuanced management of different protocol
types within the `flow_cache`, we've leveraged the newly added feature in
xhash. This enhancement includes the introduction of Protocol-Based
Least Recently Used (LRU) caches, which substantially improves the handling
of multiple protocol types in the cache.
With this implementation, during various pruning sessions (idle,
excess, and timeout etc), we've adopted a round-robin method to cycle through
the LRU lists designated for each protocol type. This methodology ensures
that every protocol, irrespective of its inherent timeout characteristics,
is given an equal opportunity for pruning and retirement from the cache.
This adjustment to `flow_cache` not only addresses the previously observed
bias during pruning sessions, especially concerning UDP flows,
but lays down a framework for more advanced and controlled data management
within the cache moving forward.

10/25/2024
Allowlist LRU
To address the need for preserving flows that have been allowlisted and are
at risk of timing out, we've introduced a configurable Allowlist LRU cache
within the flow_cache. This enhancement enables the retention of flows marked
with a whitelist verdict, preventing them from being prematurely pruned due
to inactivity timeouts. This is particularly beneficial in scenarios where
Snort ceases to observe traffic for a flow after the whitelist decision,
especially if that flow is long-lived. Without this adjustment,
such flows may be pruned by the cache upon timeout, potentially impacting
event logging at the flow’s end-of-life (EOF) due to missing
pruned flow information.

The Allowlist LRU cache is disabled by default but can be enabled by adding
allowlist_cache = { enable = true } in the stream configuration.
Like the protocol-based LRUs, this allowlist functionality is an
additional LRU rather than a new hash_table, thereby maintaining
consistent performance with previous configurations.

04/07/2025
Changed the flow data hash to start with 7 hash buckets. This is trying to balance
between memory usage and speed.
The flow data hash load factor controls the when a hash table rebalance happens. We
don't ever want a hash rebalance to happen for flow data as that would mean allocating
a new bigger bucket array and rehashing all of the elements in the hash. So, a max
load factor of 5.0 was chosen. This means that the average depth for all buckets in
the hash would need to be 5 or more before a rebalance would happen. This is very
unlikely to happen.
