Skip to content

Conversation

@nathanneike
Copy link
Contributor

Add lazy EMD solver with on-the-fly distance computation

  • Implement emd_c_lazy in C++ network simplex for memory-efficient OT
  • Add lazy mode to emd2() accepting coordinates (X_a, X_b) instead of cost matrix
  • Support sqeuclidean, euclidean, and cityblock metrics
  • Add restrict for SIMD optimization
  • Remove debug output from network_simplex_simple.h
  • Add tests for lazy solver and metric variants

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Motivation and context / Related issue

Addresses memory limitations when computing OT with large point clouds. Instead of pre-computing and storing the full n×m cost matrix, the lazy solver computes distances on-the-fly during the network simplex algorithm. This reduces memory from O(nm) to O(n+m) while maintaining exact EMD solutions.

How has this been tested

  • All existing tests pass (1031 passed, 69 skipped)
  • Added new tests in test_solvers.py: test_solve_sample_lazy and test_solve_sample_lazy_emd
  • Verified correctness against standard EMD solver
  • Tested all three metrics (sqeuclidean, euclidean, cityblock)
  • Confirmed SIMD optimization with compiler output analysis

PR checklist

  • I have read the CONTRIBUTING document.
  • The documentation is up-to-date with the changes I made (check build artifacts).
  • All tests passed, and additional code has been covered with new tests.
  • I have added the PR and Issue fix to the RELEASES.md file.

- Implement emd_c_lazy in C++ network simplex for memory-efficient OT
- Add lazy mode to emd2() accepting coordinates (X_a, X_b) instead of cost matrix
- Support sqeuclidean, euclidean, and cityblock metrics
- Add __restrict__ for SIMD optimization
- Remove debug output from network_simplex_simple.h
- Add tests for lazy solver and metric variants
@rflamary rflamary changed the title Add lazy EMD solver with on-the-fly distance computation Add lazy EMD solver with O(n) memory requirement Jan 20, 2026
@codecov
Copy link

codecov bot commented Jan 20, 2026

Codecov Report

❌ Patch coverage is 91.45299% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.97%. Comparing base (4c49769) to head (58c520c).
⚠️ Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (4c49769) and HEAD (58c520c). Click for more details.

HEAD has 19 uploads less than BASE
Flag BASE (4c49769) HEAD (58c520c)
20 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #788       +/-   ##
===========================================
- Coverage   97.07%   84.97%   -12.10%     
===========================================
  Files         107      107               
  Lines       22156    22249       +93     
===========================================
- Hits        21507    18906     -2601     
- Misses        649     3343     +2694     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator

@rflamary rflamary left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly there, this is a wonderful PR that will allows solvers on very large data (at the cost of computing time)

metric="sqeuclidean",
numItermax=100000,
log=False,
return_matrix=False,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since it is sparse, it could be true by default

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The solver here is only for dense ? So not sure why it should be true by defautl

@rflamary rflamary merged commit 26f1380 into PythonOT:master Jan 22, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants