Skip to content

Conversation

@musaqlain
Copy link

@musaqlain musaqlain commented Dec 25, 2025

Description

The current schema in conf/schema.py was restricted to List[str]. This caused omegaconf.errors.ValidationError when users tried to provide detailed augmentation configurations (dictionaries with parameters), as described in the issue.

Changes Made

  • Updated augmentations type hint in TrainConfig and ValidationConfig from List[str] to List[Any].
  • Added a new regression test test_augmentation_schema_validation in tests/test_augmentations.py to ensure dictionary configurations are accepted.

Testing

  • Added test_augmentation_schema_validation which explicitly initializes a model with a complex dictionary augmentation list (e.g., RandomResizedCrop with params) and asserts it is stored correctly without raising a ValidationError.
  • Ran pytest tests/test_augmentations.py to ensure existing string-based augmentations (like ["HorizontalFlip"]) still pass.
  • Verified locally with a reproduction script.

Linked Issue

Closes #1234

AI-Assisted Development

I utilized AI for sanity checking code logic and conducting an assisted review to identify potential missing resets and schema constraints.

  • I used AI tools (e.g., GitHub Copilot, ChatGPT, etc.) in developing this PR
  • I understand all the code I'm submitting

@jveitchmichaelis
Copy link
Collaborator

jveitchmichaelis commented Jan 1, 2026

Thanks for this, please could you provide test cases that would cover the changes? e.g. configs with (dict) augmentation listings that would currently fail.

Also as the other PR, please could you also include the AI assistance declaration from the template?

Copy link
Contributor

@henrykironde henrykironde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@musaqlain, We expect some tests for changes in logic.
Thank you for the contribution

@codecov
Copy link

codecov bot commented Jan 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.83%. Comparing base (0ab23a3) to head (1448db0).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1248      +/-   ##
==========================================
+ Coverage   87.73%   87.83%   +0.09%     
==========================================
  Files          20       20              
  Lines        2716     2720       +4     
==========================================
+ Hits         2383     2389       +6     
+ Misses        333      331       -2     
Flag Coverage Δ
unittests 87.83% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@musaqlain
Copy link
Author

@musaqlain, We expect some tests for changes in logic. Thank you for the contribution

thanks, I have added the requested test case in tests/test_augmentations.py. now it verifies that the new config schema accepts dictionary based arguments format...

@musaqlain musaqlain requested a review from henrykironde January 8, 2026 00:22
Copy link
Collaborator

@jveitchmichaelis jveitchmichaelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding that! In the test case, please pass the dict to utilities.load_config or as a config_arg to main.deepforest.

The current test assigns the dict after instantiation and I'm not sure if the schema is re-validated? Maybe it does, but it's a bit clearer if we check how the code would normally be used (ideally we should set up the config once and not modify it).

@musaqlain
Copy link
Author

Thanks for the feedback! You are right, testing it during initialization is much safer to ensure validation triggers correctly. I have updated it.

Copy link
Collaborator

@jveitchmichaelis jveitchmichaelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@musaqlain thanks, I will take a closer look this week. I added a couple more suggestions, while we're adding this fix we could take a moment to improve the tests more.

The test should cover some other variations that we might typically expect; for example it's valid to mix strings ("HorizontalFlip", to use defaults) and dicts (to customize parameters). Note that we do test the parser for this behaviour earlier in the file.

Another suggestion is to enforce stricter typing here, as Any is very broad. I need to check how OmegaConf handles unions, but we could try list[union[str, dict]] or something similar. I'm not sure if we would also have to add a specific type for the dict.

@musaqlain
Copy link
Author

musaqlain commented Jan 9, 2026

Thanks for the detailed review! I've updated the PR with the following changes:
1. Robust Integration Test:
I completely refactored test_augmentation_schema_validation as suggested. It now:

  • Defines a mixed list of augmentations (String + Dictionary) to ensure both formats work together.
  • Explicitly loads a dataset using these augmentations.
  • inspects the actual transforms pipeline to assert that the correct Kornia objects (e.g., K.RandomResizedCrop) are instantiated.
  • also moved this test function at the top of file along with other integration tests.

2. Schema Typing:
I attempted to enforce stricter typing using list[str | dict[str, Any]] (or Union) as requested. However, OmegaConf (Structured Configs) currently raises a ValidationError: Unsupported value type when using Unions for mixed lists.
To ensure stability, I have reverted the schema to list[Any], but I added the integration test above to guarantee that valid dictionary configurations are processed correctly.

This is a known limitation in OmegaConf where Union types are not yet fully supported in complex Structured Config containers (see OmegaConf Issue #144 and Issue #1166).

@musaqlain
Copy link
Author

Hi, does it look good? I have updated the integration tests. can you please check now. cc @jveitchmichaelis @henrykironde

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How should I list the augmentations in the config.

3 participants