Abstract

Linking postmortem histopathology with MRI is essential for validating neuroimaging biomarkers at the cellular level. However, registering single histology sections to 3D MRI volumes presents compounded challenges: localizing the correct slice plane within the volume, solving a part-to-whole problem caused by tissue fragmentation, and bridging fundamental differences in resolution and contrast between modalities. We present an end-to-end framework using implicit neural representations (INRs) that jointly addresses these challenges through three coordinated SIREN networks: a support network that extracts shared anatomical structure, a residual network that captures stain-specific appearance, and a deformation network that models tissue displacements. A two-phase optimization first learns global affine alignment and image decomposition, then estimates local, non-rigid, and 3D deformations with frozen affine parameters. We validate this pipeline on 18 postmortem MRI-histology pairs with 7T T2 MRI and LFB-CV stained histology slides. Our method achieves the highest robustness across initialization conditions (94–100% success rate vs. 67–100% for baselines) while maintaining comparable accuracy to established optimization-based methods on successful cases. This joint optimization avoids error accumulation from sequential pipelines and enables accurate, high-throughput registration suitable for large-scale clinico-pathological studies.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6039_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/cathalye/histo_mri_INR_registration

Link to the Dataset(s)

N/A

BibTex

@InProceedings{AthChi_Joint_MICCAI2026,
        author = { Athalye, Chinmayee AND Trotman, Winifred AND Levorse, Lisa M. AND Teunissen-Bermeo, Eric AND Capp, Noah AND Emrani, Sheina AND Ittyerah, Ranjit AND Prabhakaran, Karthik AND Robinson, John L. AND Lee, Edward B. AND Ohm, Daniel T. AND Tisdall, M. Dylan AND Das, Sandhitsu R. AND Wolk, David A. AND Irwin, David J. AND Yushkevich, Paul A.},
        title = { { Joint Decomposition and Learned Deformation for Part-to-Whole MRI to Histology Registration } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16889},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    Registering brain histology with MRI can be difficult as brain histology is prone to tissue deformations. To reduce the deformations, the paper uses patient specific 3D printed molds during tissue sectioning, and acquires thicker sections. The sectioned tissues are stained with LFB-CV, which stains histology “similarly” as it would appear on MRI. The histology is decomposed using “S” (shared anatomy that captures MRI-consistent anatomical features) and “R” (stain-specific residual that captures stain-specific features) networks to get the support image “S(x)” that now looks even more like MRI. This image along with the original histology are jointly used to optimize the alignment of the corresponding MRI slice, M(\Phi(x)) (where \Phi is predicted by “D”).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Study design seems to be well thought out in terms of using patient specific molds and LFB-CV. Figure 1 effectively illustrates the proposed pipeline. The idea of decomposing histology using “S” and “R” networks is interesting and the different loss functions are well explained. S-RMSD is listed for variety of other methods for 18 cases.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The dataset section 2.1 could include more details (e. g. The number of subjects is only later mentioned in the results section. The introduction mentions the paper used brains with Alzheimer’s but the data section has no mention of Alzheimer’s). While the qualitative results for each branch are provided in Figure 3, an ablation study could help readers quantify the improvement. Figure 3 also shows how MRI is transformed to improve alignment between MRI and histology. The paper states that “the learned deformation field captures meaningful tissue distortions. ” Histology is more prone to processing artifacts and these artifacts are not always biologically meaningful or reflective of the anatomical structure. The earlier sentence can imply (i) histological deformation is biologically meaningful tissue distortion or (ii) histology provides better ground truth in terms of structure. It is unclear what the benefit of deforming (often less deformed) MRI to look like already deformed histology is. The results could be further elaborated and additional context about the data would improve interpretability.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    There appears to be repeated citation of the same authors across multiple references (e. g. [3], [4] and [12], [13], and [16], [17]). While this might provide a more focused overview of the previous work, it is always nice to provide a broader coverage of related work. The authors could also consider reporting average +/- standard deviation instead of ranges (e. g. “Our method achieves the highest robustness across initialization conditions (94–100% success rate vs. 66–100% for baselines)”)

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Registering histology with other medical imaging modalities can be insightful yet challenging. The methodology seems interesting and the data acquisition (e.g. section thickness, patient specific molds, LFB-CV staining) seems to be well thought out. The paper looks into robustness (with/without manual initialization) and S-RMSD for the proposed method and other existing methods. The paper is missing a few details on the dataset and the dataset size seems to be relatively small. Qualitative statements feel a little overstated and additional analysis such as ablation studies and clearer interpretation of the learned deformation fields could strengthen the paper. Overall, the paper is promising but the evaluation is somewhat limited in scope.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The problem itself is interesting, and I think the methodological setup is sufficient. Although I still have some reservations about the direction of registration, phrasing, and other minor things, the overall contribution warrants an accept.



Review #2

  • Please describe the contribution of the paper

    The authors aimed to develop an implicit neural representation (INR) based framework for the registration of post-mortem brain MRI to a histological slide from the same brain. The method uses two phases; the first phase involves training an INR network to decompose the histology slide into anatomical (greyscale) and stain-specific (coloured) representations, and learning the correct oblique plane of the MRI sub-volume the histology slide belongs to. The second phase attempts to find the deformation field mapping the MRI to the histological slide while fixing the plane selection and further refining the image decomposition. The authors achieve results comparable to state-of-the-art methods.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is tackling an important and timely problem for neuroimaging
    • Combines ultra-high-resolution 7T imaging and histology
    • Acquisition of such data is complex
    • Interesting application of implicit neural representations
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Major Comments

    • While the methods presented in this work are interesting, the selection of 8 hyperparameters for loss functions alone and extreme scheduling (e.g. scheduling L_FTV from 1 to 50) of those hyperparameters lends itself to overfitting the specific dataset and these parameters likely will not generalize to new data.
    • The methodology presented in this work does not represent a real improvement when compared to state-of-the-art methods in-spite of being framed as such. Results should be presented both with and without the outliers to get a better understanding of overall performance, as well as include what their definition of an outlier is.

    Minor Comments

    • The methodology developed in this work is not presented in its clearest form. o The workflow shown in Figure 1.B is unclear as it does not show what each network takes as inputs and what it produces as outputs. It is also unclear as to what is trainable in phase 1 and phase 2.The authors should consider recreating the figure to show phase 1 and 2 separately, as well as illustrate all model’s inputs and outputs while keeping the relationships between each of the models. o A more detailed description of how “fine-grained initialization” was conducted should be added. o The evaluation methodology is also unclear. What is the S-RMSD being measured between? The drawn contour and resampled contour? Was the contour redrawn on the histological image and the S-RMSD computed between the resampled and ground truth histological contour?
    • Model architectures should be presented in a figure to best show their structure.
    • The authors should consider conducting failure analysis to investigate why some models failed to yield a coherent registration
    • The authors should consider investigating performance depending on the anatomical location of the tissue (e.g. does temporal lobe registration perform better than parietal lobe registration)
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors present a framework that has 8 different loss function terms, each with their own weighting, some of which are scheduled during the training procedure. This does not even include the other hyperparameters in the model. These likely overfit to their specific dataset and task and do not represent a real generalizable framework, and would likely need to be refit for new data that is acquired and stained somewhere else. All of this was done to not achieve significantly better performance than their comparison methods.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors propose an implicit neural representation (INR) framework for registering 2D postmortem histology sections to 3D MRI volumes. Three SIREN networks jointly perform image decomposition (shared anatomy vs. stain-specific appearance) and 3D deformable registration.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Co-optimizing decomposition and registration so that appearance disentanglement and alignment mutually inform each other is conceptually clean and avoids the error propagation inherent in sequential style-transfer + registration pipelines.

    • The tissue preparation is rigorous and well explained.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Imprecise and inconsistent terminology:

    • The displacement field is repeatedly called a “flow” but, in the context of registration, flows are aimed to be integrated to produce a diffeomorphic transformation which is not the case here (see: A log-Euclidean framework for statistics on diffeomorphisms, Arsigny 2006).
    • The term “folding” is used in contradictory ways: the abstract states the method models tissue folding, while the Jacobian regularizer is meant to prevent it. Neither usage is clarified.

    Non-motivated or imprecise design choices:

    • The final transformation is defined as phi(x) = x_affine + flow(x). It should be justified why addition is used instead of transformation composition.
    • Why not integrating the flow to promote diffeomorphic transformations?
    • Why grayscale enforcement via inter-channel penalty rather than a single-channel output?
    • Why a background variance loss to promote a constant value outside the tissue mask rather than just masking with the tissue mask?
    • Why LCC between moved MRI and fixed histology, and not only between moved MRI and support? It’s the whole point of a support, so we don’t have to compare images with contrasts that are too different.
    • How is the S-RMSD metric computed? nearest neighbor?

    The above-mentioned choices lead to having as many as 8 different losses, which is huge for the task.

    The plotting of the outliers is confusing, why are they all capped to a given value?

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The core idea is sound and the tissue preparations seem rigorous. However, there are several foundational aspects of the computational method that are either imprecisely described or insufficiently justified.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have mostly clarified the design choices I had considered non-motivated or imprecise and the imprecise or inconsistent terminology. Providing they also include those informations in the camera-ready version, the paper could reach the threshold for MICCAI acceptance. The huge amount of losses is still somewhat concerning however.



Author Feedback

We thank the reviewers & AC. Reviewers found the method important & timely (R2), well thought out (R1), with rigorous tissue preparation (R4), our joint decomposition-registration formulation conceptually clean (R4). We address specific concerns:

METHOD CLARIFICATIONS (AC, R1, R2, R4) Non-rigid deformation stage produces nearly diffeomorphic transformations with mean Jacobian determinant of 1.002 (±0.21). Only 0.28% of points exhibit -ve Jacobians; 5 of 18 image pairs are fully diffeomorphic (no -ve Jacobians). Visualizations will be included in camera-ready supplementary material.

Flow vs displacement, total variation (R4, AC): In registration, flow denotes (e.g. stationary) velocity field integrated to a diffeomorphism, which we do not perform. We will replace “flow” with “displacement” throughout & rename L_ftv. The AC rightly notes TV uses an L1 norm; ours is L2.We will clarify this.

φ = x_affine + flow(x) (R4): Reflects the two serial phases (affine, deform) in our model. We will switch to composition per standard notation; we do not expect the optimal solution to change.

“Folding” is contradictory (R4): It holds two distinct meanings - physical tissue “folding”/ movement from histology processing modeled by deformation network D (Sec 2.3 last line); topological folding of the transformation reduced by L_reg (Sec 2.4 Jacobian). We will clarify this distinction.

LOSSES (R1, R2, R4) Hyperparameter generalization (R2): The same loss weights & schedules were applied across all 18 cases without per-pair tuning. The alphas are loss weights setting the relative importance of each loss, with rationale provided for each term. As meta-review notes, INRs optimize network weights (not hyperparameters) per-pair; it’s a strength, not a limitation. We acknowledge the lack of a held-out validation set for hyperparameters as a limitation.

L_cc against both fixed & support (R4): Registration needs grounding while support network S is training. Early in Phase 1, S is uninformative, so L^S_cc cannot drive registration alone. L_cc against the fixed histology anchors registration until S matures; alpha^S_cc is ramped 1 to 2 (Sec 2.4) so support L^S_cc dominates once S is trained well.

Grayscale penalty, not single-channel S (R4): A single-channel S would still need 3-channel broadcasting for additive reconstruction of fixed image F ≈ S + R. Having S generate 3-channel image with grayscale penalty, allows to have identical support S & residual R architectures to simplify decomposition.

Background variance loss vs. masking (R4): We do mask - the loss is computed only outside mask Ω. Without it, S models background staining artifacts stochastically; we require artifacts confined to R, so S must be uniform outside the tissue.

Ablations (R1): Will be included in camera-ready supp material.

DATASET (R1) Full data acquisition protocol is in a companion methods paper (anonymized). We have 1000+ pairs but evaluation is bottlenecked by manual contour annotation. Future work will use automated GM/WM Dice on a larger cohort.

EVALUATION & RESULTS (R1, R2, R4, AC) S-RMSD definition (R2, R4): Symmetric nearest-neighbor RMS distance between the manually-traced fixed-histology contour & the resampled MRI contour after registration, both in histology space.

Fig 2 outliers (R2, R4): The plot conflates failed registrations (no valid S-RMSD) with successful-but-poor cases. We will separate them clearly.

Fig 1 clarity (R2, AC): We will split Phase 1 & Phase 2 panels, explicitly label inputs, outputs, trainable components per phase (already specified in Sec 2 & 2.3; the figure will mirror the info).

Accuracy vs robustness (R2): Our claim is robustness with comparable accuracy, not raw-accuracy gains - critical for high-throughput use without manual intervention. INR achieves 94% success on automated init vs. 88% (greedy), 66% (TIRL), is the only method whose deformable stage significantly improves over its affine stage (p=0.012 manual, p=0.017 auto).




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers had mixed opinions on the manuscript ranging from Weak Accept, Weak Reject, and Reject (4, 3, 2). Two of the reviewers were somewhat or not certain with their assessment while the reviewer scoring with a WR was very confident.

    Overall, the application is very relevant to Miccai community and the complexity of the proposed method relies on the problems with histological image acquisition, which may break the topology between the two anatomies. Indeed, it is a 2D – 3D registration problem in its origin, which requires finding first the correct oblique slice plane. It is remarkable that the pipeline of the method is not simply registering two images, but it involves several complicated steps. Copying the paragraph from the manuscript:

    “… localizes the correct oblique slice plane within M , locates the subregion of the histology slide in the coronal slice, and models regional, out of plane tissue deformations introduced during histological processing.”

    The complexity of the problem is translated to the method. Although it is based on a simple INR in the small deformation setting, it needs up to 8 different losses to properly work. It is important to remark that the INR is treated as a pairwise method, therefore, for each image pair we have to learn the parameters from scratch of a new MLP.

    Having said this, I believe that the manuscript would greatly benefit from a rebuttal period. I would advise the authors to work on the most relevant problems raised by the reviewers, trying to fix, clarify, or justify all comments (many of them are minor comments that should be addressed in the manuscript). It would be also very interesting to show an image of the deformation grids in order to better assess the quality of the computed deformations (in the final non-rigid deformation stage, after the affine alignment) and to provide metrics such as the number of negative Jacobians, maximum, and minimum.

    One final minor comment, the term “Flow total variation”… the term total variation is related with L1 norm.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After the rebuttal phase, the reviewers agreed in the acceptance of the manuscript. The overall contribution is of relevance to Miccai community. The camera ready manuscript should include most issues pointed out by the reviewers to agree with standard quality Miccai publications. One of the reviewers commented a concern on warping structurally intact anatomy to match processing histological artifacts and the sentence “the learned deformation field captures meaningful tissue distortions”. In addition, the paper would benefit from explicitly discussing the directional choice.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    This work proposes an approach to register MRI with histology. The approach makes use of three implicit neural representations to 1/2) decompose the histology image additively into a component that is similar to the MR image and a component that contains stain-specific appearance, and 3) to capture the spatial deformation.

    The approach is interesting but, in my opinion, in its current form not sufficiently validated. The approach depends on a very large number of hyperparameters (there are 8 different loss term that appear to have been tuned without a separate validation data set; this concern was highlighted by two of the reviewers as well as meta reviewer 1) and has only been tested on 18 MRI slabs (possibly from the same MRI, though this has not been clearly specified).

    Further, while the rebuttal indicates that the focus is on robustness, the robustness improvements appear to be marginal (with 17/18 successes for the proposed approach and 16/18 for greedy) given the complexity of the proposed approach.

    In my opinion, this is interesting work, but it is not quite ready for MICCAI yet.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The problem you are addressing is very challenging and it requires careful considerations. The reviewers had split opinions, reflecting the complexity of the task or challenges in addressing it. I recommend acceptance of the study based on the overall quality and reviewers comments, but I strongly encourage you to consider the critical point related to overfitting which is very common in registration approaches.



back to top