Skip to content

Handling of 'optional' fields in source data sets #77

@gyoliver

Description

@gyoliver

Issue:
The BaseModel method "reload_sources" requires that you pass in "skip" for the source dataset's population field when instantiating any model, even those models do not require population data (i.e., AccessModel, AccessCount, AccessTime, and AccessSum).

Similarly, the BaseModel method "reload_dests" makes the same requirement for a capacity field in the destination dataset, while AccessModel, AccessCount, AccessTime don't require capacity data.

If you don't pass in "skip" for these fields, the code throws an error. I think users will go on the assumption that you only have to pass in/deal with fields that are required for a model, so this could be confusing. It took me a while to pinpoint what the issue was. Debugging is also confusing since (as of v0.1.10), the errors thrown are quite general--SourceDataNotParsableException and DestDataNotParsableException.

The lines in v0.1.17 triggering the error are:

  • line 294: if self.source_column_names['population'] == 'skip':
  • line 380: if self.dest_column_names['capacity'] == 'skip':

Recommended modifications:
Could we modify the logic so it doesn't require passing in "skip" for these fields?
Also, it'd be good to replace the "skip" value with None and/or blank string.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions