Formats API Reference¶

The formats module provides reliable, tested functions for loading and saving event data in various file formats.

Overview¶

import evlib

The formats module supports: - Text files: Space-separated event data with flexible column mapping - HDF5 files: Hierarchical data format with perfect round-trip compatibility - Advanced filtering: Time windows, spatial bounds, polarity selection

Core Functions¶

load_events¶

::: evlib.load_events

The primary function for loading event data with comprehensive filtering options.

Example Usage:

# High-level API (recommended) - returns Polars LazyFrame
events = evlib.load_events("data/slider_depth/events.txt")
df = events.collect()

# Time window filtering using evlib.filtering
import evlib.filtering as evf
events = evlib.load_events("data/slider_depth/events.txt")
events_df = events.collect()
filtered_events = evf.filter_by_time(events_df, t_start=0.0, t_end=1.0)

# Spatial filtering using filtering functions
roi_filtered = evf.filter_by_roi(events_df, x_min=100, x_max=500, y_min=100, y_max=300)

# Polarity filtering using Polars
import polars as pl
events = evlib.load_events("data/slider_depth/events.txt")
positive_events = events.filter(pl.col('polarity') == 1)

# Data is sorted by time by default when using evlib.load_events

Parameters: - path (str): Path to the event file - t_start (float, optional): Start time filter (inclusive) - t_end (float, optional): End time filter (inclusive) - min_x (int, optional): Minimum x coordinate (inclusive) - max_x (int, optional): Maximum x coordinate (inclusive) - min_y (int, optional): Minimum y coordinate (inclusive) - max_y (int, optional): Maximum y coordinate (inclusive) - polarity (int, optional): Polarity filter (1 for positive, -1 for negative) - sort (bool): Sort events by time after loading

Returns: - polars.LazyFrame: A Polars LazyFrame with columns x, y, t, polarity. Call .collect() to materialise a DataFrame.

save_events_to_hdf5¶

::: evlib.save_events_to_hdf5

Save event data to HDF5 format with perfect round-trip compatibility.

Example Usage:

import evlib
import numpy as np

# Prepare event data with correct dtypes
xs = np.array([320, 321, 319], dtype=np.int64)
ys = np.array([240, 241, 239], dtype=np.int64)
ts = np.array([0.1, 0.2, 0.3], dtype=np.float64)  # in seconds
ps = np.array([1, -1, 1], dtype=np.int64)  # Use -1 for negative polarity

# Save events to HDF5
output_path = "output_example.h5"
evlib.save_events_to_hdf5(xs, ys, ts, ps, output_path)
print(f"Saved {len(xs)} events to {output_path}")

# Note: HDF5 round-trip currently has type compatibility limitations
# The save function converts to uint16/int8 internally for space efficiency
# To load saved HDF5 files, use the standard evlib.load_events() function
# which handles the type conversions automatically

save_events_to_text¶

::: evlib.save_events_to_text

Save event data to text format.

Example Usage:

import evlib
import numpy as np

# Prepare event data
xs = np.array([320, 321, 319], dtype=np.int64)
ys = np.array([240, 241, 239], dtype=np.int64)
ts = np.array([0.0001, 0.0002, 0.0003], dtype=np.float64)  # in seconds
ps = np.array([1, 0, 1], dtype=np.int64)

# Save events to text file
evlib.save_events_to_text(xs, ys, ts, ps, "output.txt")

File Format Support¶

Text Files¶

Standard Format:

# t x y polarity
0.000100 320 240 1
0.000200 321 241 -1
0.000300 319 239 1

Custom Column Mapping:

# For files with different column order, preprocessing is recommended
# x y polarity t
# 320 240 1 0.000100
# Note: Use evlib.load_events with standard format for best compatibility
events = evlib.load_events("data/slider_depth/events.txt")  # Handles standard formats
df = events.collect()
# Access data: df['x'], df['y'], df['t'], df['polarity']

HDF5 Files¶

HDF5 files are automatically detected and loaded. The format supports: - Perfect round-trip compatibility - Efficient storage and loading - Multiple dataset structures (auto-detected)

Supported HDF5 Structures:

# Structure 1: datasets inside "events" group
# Example paths: /events/t, /events/x, /events/y, /events/p

# Structure 2: separate root datasets
# Example paths: /t, /x, /y, /p
# Alternative names: /time, /x_pos, /y_pos, /polarity
# Short names: /ts, /xs, /ys, /ps

Advanced Features¶

Filtering Combinations¶

All filters can be combined:

# Complex filtering example using multiple filters
import evlib.filtering as evf
import polars as pl

# Load events and apply multiple filters
events = evlib.load_events("data/slider_depth/events.txt")
events_df = events.collect()

# Apply time filtering
time_filtered = evf.filter_by_time(events_df, t_start=1.0, t_end=5.0)

# Apply spatial filtering (ROI)
roi_filtered = evf.filter_by_roi(time_filtered, x_min=100, x_max=500, y_min=100, y_max=400)

# Apply hot pixel filtering
processed_events = evf.filter_hot_pixels(roi_filtered, threshold_percentile=99.9)

# Then filter by polarity using Polars
positive_events = processed_events.filter(pl.col('polarity') == 1)

Performance Optimization¶

import evlib.filtering as evf

# For large files, use filtering for efficiency
events = evlib.load_events("data/slider_depth/events.txt")
events_df = events.collect()
filtered_events = evf.filter_by_time(
    events_df,
    t_start=0.1, t_end=1.0  # Only load events in time range
)

Error Handling¶

try:
    events = evlib.load_events("data/slider_depth/events.txt")
    df = events.collect()
except FileNotFoundError:
    print("File not found")
except OSError as e:
    print(f"Invalid file format: {e}")

Performance Characteristics¶

Operation	Performance	Notes
Text loading	~1.2x Python	Comparable to pandas
HDF5 loading	~0.9x Python	Slight overhead for filtering
Large files (>1M events)	~1.5x faster	Memory efficiency advantage
Filtering	Very fast	Applied during loading, not post-processing

Best Practices¶

File Format Selection¶

Text files: Human-readable, easy debugging, cross-platform
HDF5 files: Faster loading, smaller file size, metadata support

Performance Tips¶

Use HDF5 for large datasets (>100k events)
Apply filters during loading rather than post-processing
Use time window filtering for memory efficiency with large files
Sort only when necessary (adds processing time)

Error Prevention¶

Validate file existence before loading
Check file format matches expected structure
Use try-catch blocks for robust error handling
Verify array shapes after loading

Migration from Other Libraries¶

From other libraries¶

# evlib provides a unified API for event data loading
import evlib
import evlib.filtering as evf

# Load events as Polars LazyFrame
events = evlib.load_events("data/slider_depth/events.txt")
df = events.collect()

# Apply filtering
start_time = 0.1
end_time = 1.0
events = evlib.load_events("data/slider_depth/events.txt")
events_df = events.collect()
filtered_events = evf.filter_by_time(events_df, t_start=start_time, t_end=end_time)