probes — Data Auditing Probes

class brewery.probes.MissingValuesProbe

Data quality statistics for a dataset field

Attributes :
  • count: total count of null records
probe(value)

Probe the value.

class brewery.probes.StatisticsProbe

Data quality statistics for a dataset field

Attributes :
  • min - minimum value found
  • max - maxumum value found
  • sum - sum of values
  • count - count of values
  • average - average value
class brewery.probes.DistinctProbe(threshold=None)

Probe for distinct values.

class brewery.probes.StorageTypeProbe

Probe for guessing field data type

Attributes:
  • field: name of a field which statistics are being presented
  • storage_types: found storage types
  • unique_storage_type: if there is only one storage type, then this is set to that type
unique_storage_type

Return storage type if there is only one. This should always return a type in relational databases, but does not have to in databases such as MongoDB.

Previous topic

Node Reference

Next topic

Data Quality

This Page