Docstring for API documentation + purity PIE for z and wavelength calibration by cliulucien · Pull Request #120 · PtyLab/PtyLab.py

cliulucien · 2026-10-08T10:31:42Z

Summary

This PR improves the documentation and organization of the PtyLab reconstruction engines.

The main goal is to make the engine API easier to understand by documenting the underlying reconstruction strategies, clarifying engine-specific parameters and update rules, and reducing duplicated legacy implementations.

Main changes

Engine documentation

Expanded and revised docstrings for the reconstruction engines, including:

purityPIE
- documented purity-based axial-distance and wavelength calibration
- documented fixed-grid and adaptive search strategies
- clarified mixed-state probe purity and calibration workflow
- added relevant literature references
aPIE
- documented detector-angle calibration and momentum-assisted angle search
- clarified the detector-coordinate transformation and reconstruction flow
- added the original aPIE reference
e3PIE
- documented the multislice reconstruction model
- described inter-slice angular-spectrum propagation
- clarified forward propagation and backward slice updates
OPR
- documented Orthogonal Probe Relaxation and position-dependent probe reconstruction
- described the low-rank probe subspace formulation
- clarified the distinction between mixed-state probe modes and position-dependent probe variations
- added relevant OPR and mixed-state OPR references
- documented the current GPU/CuPy requirement
qNewton
- documented the locally weighted and regularized update rules
- marked the current implementation as experimental
mqNewton
- documented the qNewton-style updates combined with momentum acceleration
- documented momentum, ADAM-inspired, and NADAM-inspired update schemes

Legacy engine cleanup

multiPIE has been consolidated into mPIE.

Mixed-state reconstruction is already supported through the general
reconstruction dimensions such as:

reconstruction.npsm
reconstruction.nosm
reconstruction.nlambda

The former multiPIE implementation largely duplicated the regularized
object/probe updates and momentum scheme already available in mPIE.

A backward-compatible multiPIE wrapper is retained so that existing code
using Engines.multiPIE continues to work while emitting a deprecation
warning.
Legacy parameter aliases are preserved:

betaM -> feedbackM
stepM -> frictionM

Tests

Added tests for the deprecated multiPIE wrapper, including:

inheritance from mPIE
deprecation warning
legacy parameter aliases
preservation of momentum-acceleration behavior
public exposure through Engines.multiPIE

The purityPIE adaptive and fixed calibration logic is also covered by
tests using synthetic objective functions without running full
ptychographic reconstructions.

Notes / follow-up work

During the documentation pass, several implementation details were identified
that should be handled separately and are now mentioned in different issues.

% EDIT (Shantanu): Moved those specific implementation questions and such to github issues to better track and follow-up.

…sues: external initialization

… exact same initialization of each z.

…/PtyLab.py into Add_purity_axial_scan

…arison - add fixed and adaptive wavelength calibration - add local quadratic fitting of purity scan maxima - store and plot reconstruction error alongside probe purity - unify scan plotting for z and wavelength calibration - update the example script for selectable z or wavelength calibration

minor change in reconstruction.py: the old style np.int.

…branch

1. OPR engine documentation 2. remove redundant multiPIE engine, added a wrapper in mPIE. 3. add test for the wrapper. 4. minor change of pytest name.

ShantanuKodgirwar · 2026-10-08T12:31:50Z

        # map a change in positions to a change in z. Experimental, do not use
        self.map_position_to_z_change = False

+        # purity based Zscan


Isn't the purityPIE part of the other PR? @cliulucien

yes, this one only for the documentation. I merged that purityPIE to this branch earlier

Generally, it is not a good idea to add things that logically belong to another PR. This PR should focus only on exactly what it is doing here. If you absolutely need things from that another branch/PR then you could either add only those things here or cherry-pick commit (a bit annoying to work with though) or simply put a specific PR in draft and let the other one merge first and then you sync this particular branch/PR with main.

ShantanuKodgirwar · 2026-10-09T09:28:11Z

Yeah this file (3K lines) does not belong in this PR for instance. I will merge the other PR first.

…ment, instead either explicit path if specified or simple input() command

… linting

…ely if it is not bell shaped which is a complete and total possibility

…ot a parameter per say

… evaluating left and right; previously it always concluded no improvements

… so now independent runs give identical values of z

ShantanuKodgirwar · 2026-10-12T14:29:26Z

@cliulucien

I updated this PR with quite many things including changing the example script. It now uses synthetic data if a stored dataset is not provided. But mainly the purityPIE engine was changed in these regards:

No changes as z or wavelength varies. mPIE in the end gets an No which would be different than at the start. Now the best candidate's object grid No is restored with its object and used for a standard mPIE run. Earlier saving after a calibration used to break or give mismatched sizes.
We now specify the same seed for every candidate or else re-run could change the answer.
A bug fix that compared each new point with itself, so it never saw an improvement. That is fixed now for the adaptive refinement.
There is no more params.plotPurityScan, now the purity plot gets updated with iterations and the final figure can be stored anyway with reconstruction.saveResults(snapshot=monitor) along with the hdf5 files recording purity with z. Also I feel params should not really be mixed with monitor-like utilities preferably.
Most importantly there is no fitted curve. Fitting a parabola doesn't make sense if the plot is not one and if there is no distinct peak, it can totally throw off with wrong answer. See the below image. After your runs, you could potentially fit whatever, but allowing this as a user option is a bad idea in my opinion.

ShantanuKodgirwar · 2026-10-12T14:47:36Z

A gif to show the updates happen live with iterations and in the last one, highlights the result and also get stored as a png if you pass snapshot=monitor (earlier PR that was merged to main): Ex: reconstruction.saveResults("filename.hdf5", snapshot=monitor).

@cliulucien could you please validate once with your dataset and tell me how it is. Then we can merge this PR.

ShantanuKodgirwar · 2026-10-12T15:03:00Z

@cliulucien I will close the other PR #118 and just merge this one after you validate with your dataset. The two have drifted and doesn't make sense to spend time and fix the other one. But in general please keep separate PRs for separate things and store one in draft if the other one is waiting to merge

cliulucien · 2026-10-15T11:47:19Z

The latest functions of adaptive purityPIE have been tested with an experimental dataset of biological specimens and work normally, except the probe window shows some random things, but the purity value is correct. I will check the settings a bit, but I think it can be merged already.

ShantanuKodgirwar · 2026-10-15T12:03:56Z

Nice. LGTM!

Chang Liu and others added 23 commits September 30, 2026 17:54

API Documentation - zPIE- started

7eafb2b

Implemented purityPIE for purity-based axial-distance calibration. Is…

3dd5311

…sues: external initialization

Change the backend of purityPIE to directly call mPIE, this make sure…

b99f4e8

… exact same initialization of each z.

clarified and sort the example script

c8ad573

Merge branch 'main' into Add_purity_axial_scan

95262a7

Add adaptive search function for purity-based axial calibratoin

2641779

Merge branch 'Add_purity_axial_scan' of https://github.com/cliulucien…

8550767

…/PtyLab.py into Add_purity_axial_scan

minor change: remove switch

d7c3b7c

Merge branch 'Add_purity_axial_scan' into API_Doc_Engines

77e94f9

API doc for purityPIE and zPIE engines

ef21933

minor change

fb2b1ff

minor change: delete old code.

b26292b

Merge branch 'Add_purity_axial_scan' into API_Doc_Engines

ae5e75c

doc of purityPIE and aPIE

83347e0

Update the test for purityPIE

d482260

Doc for aPIE finished

b807133

doc finished: e3PIE.

b6172b5

minor change in reconstruction.py: the old style np.int.

minor change: delete legacy test and warning messages

5d2ad2a

start doc for OPR engine

ea295d8

remove purityPIE test temporily, will be merged later from the other …

ba0097b

…branch

API Doc + cleanup:

aa9238e

1. OPR engine documentation 2. remove redundant multiPIE engine, added a wrapper in mPIE. 3. add test for the wrapper. 4. minor change of pytest name.

doc update: qNewton and mqNewton

ecc7c27

ShantanuKodgirwar reviewed Oct 8, 2026

View reviewed changes

Minor change: add doc for force in showReconstruction()

feab0f4

ShantanuKodgirwar reviewed Oct 9, 2026

View reviewed changes

ShantanuKodgirwar added 4 commits October 12, 2026 14:01

Merge remote-tracking branch 'origin/main' into API_doc_Engines

a9a8739

removing Tk reliance since it will not run on server/headless environ…

1d2e5c2

…ment, instead either explicit path if specified or simple input() command

adding the purity scan plot under params for attribute check

a55ad65

renaming file since we also allow synthetic data

43c16c4

ShantanuKodgirwar added 11 commits October 12, 2026 15:02

renaming monitor name and adding another one for live tracking + ruff…

860db24

… linting

with changing No, best candidate's grid copied back

1be627b

A quadratic fit is wrong for this and can throw off the curve complet…

c9016f5

…ely if it is not bell shaped which is a complete and total possibility

store the final png result

4deee8e

removing the option for purityScanPlot, this belongs to monitor and n…

155c835

…ot a parameter per say

plotPurityScan removed and replaces with the final monitor

7bd7dcd

monitor has an extra figure that registers its window as purity scan

d5e7792

removing the legacy parameter

7098d10

fixing the refinement bug where each round records best_before before…

1821151

… evaluating left and right; previously it always concluded no improvements

better to add a new parameter seed where every candidate starts from,…

ee9ce39

… so now independent runs give identical values of z

updating a small note

138e208

ShantanuKodgirwar added 3 commits October 12, 2026 16:50

separate hook to store the values after saveResults() is initiated

541f738

minor fix

7bfc84a

minor changes to fix docs CI

fb10685

ShantanuKodgirwar changed the title ~~API documentation for engines and cleanup legacy PIE implementations~~ Adding purityPIE and cleanup legacy implementations Oct 12, 2026

ShantanuKodgirwar changed the title ~~Adding purityPIE and cleanup legacy implementations~~ API changes + purity PIE Oct 12, 2026

ShantanuKodgirwar mentioned this pull request Oct 12, 2026

Add purity-based axial calibration with independent mPIE reconstructions #118

Closed

ShantanuKodgirwar changed the title ~~API changes + purity PIE~~ Docstring for API documentation + purity PIE for z and wavelength calibration Oct 12, 2026

ShantanuKodgirwar merged commit 55b356c into PtyLab:main Oct 15, 2026
6 checks passed

Navigation Menu

Repository navigation

Docstring for API documentation + purity PIE for z and wavelength calibration - #120

Docstring for API documentation + purity PIE for z and wavelength calibration#120
ShantanuKodgirwar merged 42 commits into
PtyLab:mainfrom
cliulucien:API_doc_Engines

cliulucien commented Oct 8, 2026 •

edited by ShantanuKodgirwar

Loading

Uh oh!

ShantanuKodgirwar Oct 8, 2026 •

edited

Loading

Uh oh!

cliulucien Oct 8, 2026

Uh oh!

ShantanuKodgirwar Oct 9, 2026 •

edited

Loading

Uh oh!

ShantanuKodgirwar Oct 9, 2026

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading

Uh oh!

cliulucien commented Oct 15, 2026

Uh oh!

ShantanuKodgirwar commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cliulucien commented Oct 8, 2026 • edited by ShantanuKodgirwar Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Main changes

Engine documentation

Legacy engine cleanup

Tests

Notes / follow-up work

Uh oh!

ShantanuKodgirwar Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cliulucien Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

ShantanuKodgirwar Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShantanuKodgirwar Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ShantanuKodgirwar commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cliulucien commented Oct 15, 2026

Uh oh!

ShantanuKodgirwar commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cliulucien commented Oct 8, 2026 •

edited by ShantanuKodgirwar

Loading

ShantanuKodgirwar Oct 8, 2026 •

edited

Loading

ShantanuKodgirwar Oct 9, 2026 •

edited

Loading

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading

ShantanuKodgirwar commented Oct 12, 2026 •

edited

Loading