five_safes_tes_analytics.node package¶

Submodules¶

five_safes_tes_analytics.node.local_processing module¶

class five_safes_tes_analytics.node.local_processing.ContingencyTable(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: BaseLocalProcessing

Build a contingency table from one or more categorical columns.

Dynamically detects columns from the user query and groups by them to count occurrences. Returns raw counts for each combination of categorical values, which can be aggregated across multiple TREs.

analysis_type = 'contingency_table'¶

property description¶: Description of the processing step.

get_columns_from_user_query()[source]¶

property processing_query¶: SQL fragment for the processing step. By default, returns None.

property user_query_requirements¶: Requirements for the user query.

class five_safes_tes_analytics.node.local_processing.Mean(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: BaseLocalProcessing

Calculate the mean of a numeric column using SQL aggregation.

Returns aggregated statistics (count and sum) that can be used to compute the mean across multiple TREs. The mean is calculated as sum/n on the client side.

analysis_type = 'mean'¶

property description¶: Description of the processing step.

property processing_query¶: SQL fragment for the processing step. By default, returns None.

property user_query_requirements¶: Requirements for the user query.

class five_safes_tes_analytics.node.local_processing.PMCC(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: BaseLocalProcessing

Calculate Pearson’s correlation coefficient between two numeric columns.

Returns aggregated statistics (count, sums, and cross-products) needed to compute PMCC across multiple TREs. The correlation is calculated on the client side after aggregating results from all TREs.

analysis_type = 'PMCC'¶

property description¶: Description of the processing step.

property processing_query¶: SQL fragment for the processing step. By default, returns None.

property user_query_requirements¶: Requirements for the user query.

class five_safes_tes_analytics.node.local_processing.PercentileSketch(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: BaseLocalProcessing

Calculate percentile sketch using TDigest algorithm.

Uses TDigest (https://github.com/CamDavidsonPilon/tdigest) to create a compact representation of the data distribution. The sketch is computed in Python after fetching all data from SQL, and returns a TDigest dictionary that can be merged across multiple TREs.

analysis_type = 'percentile_sketch'¶

property description¶: Description of the processing step.

property processing_query¶: SQL fragment for the processing step. By default, returns None.

python_analysis(sql_result)[source]¶

Optional Python-side analysis. Override in subclasses if needed. By default, does nothing and returns None.

Parameters:: sql_result – SQLAlchemy Result object from query execution
Returns:: For single result objects (e.g., {“total”: 100, “n”: 10}) list[dict]: For multiple rows (e.g., [{“gender”: “Male”, “n”: 5}, {“gender”: “Female”, “n”: 3}]) None: If no Python analysis is needed (SQL results will be used instead)
Return type:: dict

Examples

Single dict: {“centroids”: […], “count”: 100} # PercentileSketch List of dicts: [{“category”: “A”, “n”: 10}, {“category”: “B”, “n”: 20}] # Contingency tables

property user_query_requirements¶: Requirements for the user query.

class five_safes_tes_analytics.node.local_processing.Variance(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: BaseLocalProcessing

Calculate the variance of a numeric column using SQL aggregation.

Returns aggregated statistics (count, sum, and sum of squares) that can be used to compute variance across multiple TREs using the formula: Var = (sum_x2/n) - (sum/n)²

analysis_type = 'variance'¶

property description¶: Description of the processing step.

property processing_query¶: SQL fragment for the processing step. By default, returns None.

property user_query_requirements¶: Requirements for the user query.

five_safes_tes_analytics.node.local_processing.get_local_processing_registry()[source]¶

five_safes_tes_analytics.node.local_processing_base module¶

class five_safes_tes_analytics.node.local_processing_base.BaseLocalProcessing(analysis_type: str = None, user_query: str = None, engine=None)[source]¶

Bases: ABC

Abstract base class for local processing operations in federated analysis.

Each subclass represents a different analysis type that can be run on individual TREs. The class handles SQL query building and optional Python-side analysis, returning results that can be aggregated across multiple TREs.

build_query() → str[source]¶: Build a complete SQL query by combining user’s data selection with analysis calculations.

abstract property description¶: Description of the processing step.

property processing_query¶: SQL fragment for the processing step. By default, returns None.

python_analysis(sql_result: Result) → dict | list[dict] | None[source]¶

Optional Python-side analysis. Override in subclasses if needed. By default, does nothing and returns None.

Parameters:: sql_result – SQLAlchemy Result object from query execution
Returns:: For single result objects (e.g., {“total”: 100, “n”: 10}) list[dict]: For multiple rows (e.g., [{“gender”: “Male”, “n”: 5}, {“gender”: “Female”, “n”: 3}]) None: If no Python analysis is needed (SQL results will be used instead)
Return type:: dict

Examples

Single dict: {“centroids”: […], “count”: 100} # PercentileSketch List of dicts: [{“category”: “A”, “n”: 10}, {“category”: “B”, “n”: 20}] # Contingency tables

abstract property user_query_requirements¶: Requirements for the user query.

five_safes_tes_analytics.node.query_resolver module¶

class five_safes_tes_analytics.node.query_resolver.DecimalEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]¶

Bases: JSONEncoder

default(obj)[source]¶

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return super().default(o)

five_safes_tes_analytics.node.query_resolver.parse_connection_string(connection_string: str = None) → str[source]¶

Parse and convert a connection string to SQLAlchemy format.

Handles two formats: 1. Semicolon-separated: “Host=host:port;Username=user;Password=pass;Database=db” 2. Already in SQLAlchemy format: “postgresql://user:pass@host:port/db”

Parameters:: connection_string – Connection string in either format
Returns:: “postgresql://user:pass@host:port/db”
Return type:: Connection string in SQLAlchemy format

five_safes_tes_analytics.node.query_resolver.parse_semicolon_format_connection_string(connection_string: str) → str[source]¶

Parse semicolon-separated (.NET style) format connection string: Host=host:port;Username=user;Password=pass;Database=db and return a SQLAlchemy connection string. :param connection_string: Connection string in semicolon-separated format

Returns:: “postgresql://user:pass@host:port/db”
Return type:: Connection string in SQLAlchemy format

five_safes_tes_analytics.node.query_resolver.process_query(user_query, analysis, db_connection, output_filename, output_format)[source]¶

Run the given analysis (SQL + optional Python) against the DB and write results to file.

Parameters:

user_query – SQL query or user selection (used by the analysis processor to build the final query).
analysis – Analysis type (e.g. ‘mean’, ‘variance’); must exist in LOCAL_PROCESSING_CLASSES.
db_connection – Database connection string (SQLAlchemy URL, semicolon format, or None to use env vars).
output_filename – Base name for the output file (extension added from output_format).
output_format – Output format (‘json’ or ‘csv’).

Returns:

None. Writes result to disk. Exits with code 1 on error.

five_safes_tes_analytics.node.query_resolver.validate_environment()[source]¶: Validate environment variables for database connection.

five_safes_tes_analytics.node package¶

Submodules¶

five_safes_tes_analytics.node.local_processing module¶

five_safes_tes_analytics.node.local_processing_base module¶

five_safes_tes_analytics.node.query_resolver module¶

Module contents¶