Validation

Provides classes and functions to validate tracks (Track).

The scores that are calculated and evaluated are defined in Score and ScoreMetrics.

class Validator(truth_dw)

Bases: object

Class to validate Track objects with truth data.

It also has some helpers e.g. to check tracks for sanity and remove false positives.

__init__(truth_dw)

Initialization of class attributes

Parameters:truth_dw (DataWrapperTruth) – data wrapper with truth data

truth = None

DataWrapperTruth: the DataWrapperTruth with truth data

timestamps = None

list of timestamps: sorted list with all timestamps in truth

cam_timestamps = None

dict of list: sorted lists of timestamps in truth for a cam

remove_false_positives(tracks)

Removes tracks with only false positives.

Parameters:tracks (iterable of Track) – iterable structure with Track Returns:iterable structure with cleaned Track Return type:iterable of Track

sanity_check(tracks, cam_gap=True)

Performs some sanity checks on tracks.

The following checks are performed:

• No duplicate ids in tracks.
• Only one detection per timestamp
• Every detection belongs to one track only
• No unknown detections
• If cam_gap is True a track only has detections from one camera
• Each track id is unique

Parameters:tracks (list of Track) – iterable structure with Track. Keyword Arguments:  cam_gap (bool) – flag indicating that a camera switch is a insurmountable gap Returns:tuple containing:

• true_positives (int): assigned positives
• true_negatives (int): not assigned false positives
• false_positives (int): assigned false positives
• false_negatives (int): not assigned positives

Return type:tuple

validate(tracks, gap, gap_l=True, gap_r=True, cam_gap=True, val_score_fun=None, val_calc_id_fun=None, check=True)

Validates the given Track with truth data.

The default score_fun is validation_score_fun_all(). Use it as primer to implement your own.

Parameters:

• tracks (list of Track) – list of Track objects to validate
• gap (int) – the gap the algorithm should be capable to overcome

Keyword Arguments:  

• gap_l (bool) – flag indicating to consider left gap in scoring
• gap_r (bool) – flag indicating consider right gap in scoring
• cam_gap (bool) – flag indicating that a camera switch is a insurmountable gap
• val_score_fun (func) – function that takes a ScoreMetrics and calculates Score
• check (bool) – flag indicating that sanity check should be performed

Returns:

the scores for each Track as dictionary with Score objects and mapping {id => score}

Return type:

dict

score_track(track_truth, track_test, gap=0, gap_l=True, gap_r=True, cam_id=None)

Scores the equality of two Track using local alignment methods.

Parameters:

• track_truth (Track) – the first Track that is considered the source
• track_test (Track) – the second Track that is considered the copy

Keyword Arguments:  

• gap (int) – the gap the algorithm should be capable to overcome
• gap_l (bool) – flag indicating to consider left gap in scoring
• gap_r (bool) – flag indicating consider right gap in scoring
• cam_id (int) – limit the scores on one camera

Returns:

ScoreMetrics tuple with alignment information

Return type:

ScoreMetrics

validation_score_fun_all(metrics, gap=0)

Scoring function that considers all ids and gaps and calculates the percentage of matches.

Note

Used as default implementation to calculate Score.value.

Parameters:metrics (ScoreMetrics) – ScoreMetrics to calculate accumulated score Keyword Arguments:  gap (int) – the gap the algorithm should be capable to overcome Returns:value between 0 (bad) and 1 (good) indicating how good the matching is considered Return type:float

convert_validated_to_pandas(validated_results)

Converter to convert the result of a Validator to a Pandas DataFrame.

Parameters:validated_results (dict of Score) – a dictionary with Score tuples as values Returns:a Pandas Dataframe with the score information as columns Return type:pd.DataFrame

calc_fragments(dw_truth, gap, cam_gap=True)

Calculates some information about the fragments in the truth data.

Parameters:

• dw_truth (DataWrapperTruth) – DataWrapperTruth object
• gap (int) – the gap the algorithm should be capable to overcome

Keyword Arguments:  

cam_gap (bool) – flag indicating that a camera switch is a insurmountable gap

Returns:

tuple containing

• fragment_counter (int): number of fragments
• track_no_gaps_counter (int): number of tracks without gaps
• fragment_ids (int): list with number of ids per fragments
• fragment_lenghts (int): list with length of fragments

Return type:

tuple

track_statistics(tracks, scores, validator, gap, cam_gap=True)

Calculates some statistics about tracks for validation.

Parameters:

• tracks (list of Track) – iterable with tracks to analyze
• scores (dict or pd.Dataframe) – result of Validator.validate()
• validator (Validator) – Validator class to calculate track statistics
• gap (int) – the gap the algorithm should be capable to overcome

Keyword Arguments:  

cam_gap (bool) – flag indicating that a camera switch is a insurmountable gap

Returns:

some metrics about tracks

Return type:

dict

Visualization

Provides functions to visualize the validation of tracks.

Warning

The visualizations are not tested by the continuous integration server!

validate_plot(tracks, scores, validator, gap=0, cam_gap=True, metric_keys=None)

Visualizes the validation of tracks.

Arguments:

tracks (list of Track): Iterable with tracks to validate scores (dict or pd.Dataframe): Result of Validator.validate() validator (Validator): Validator class to calculate track statistics

Keyword Arguments:

gap (int): the gap the algorithm should be capable to overcome cam_gap (bool): flag indicating that a camera switch is a insurmountable gap metric_keys (list): Iterable with metric keys that should be shown.

<<<<<<< HEAD

Default is ['detections', 'fragments', 'tracks'].

Default is ['detections', 'fragments' 'tracks', 'truth_ids'].

>>>>>>> 7b6f08c50dd64fdf0248e0472874ef113bf30acf

Returns:

tuple: tuple containing figure and axes object from matplotlib

plot_fragments(scores, validator, gap, cam_gap=True, interval=5, max_x=None)

Visualizes distribution of fragments.

:param scores (dict or pd.Dataframe: Result of Validator.validate() :param validator: Validator class to calculate track statistics

Keyword Arguments:  

• gap (int) – the gap the algorithm should be capable to overcome
• cam_gap (bool) – flag indicating that a camera switch is a insurmountable gap
• interval (int) – the interval / size of bins in the histogram
• max_x (int) – upper limit for x axis

Returns:

tuple containing figure and axes object from matplotlib

Return type:

tuple