{ "version": 1, "title": "Aurora Documentation", "format": "aurora-docs-corpus", "generatedAt": "2026-07-27T19:51:19+00:00", "entryCount": 159, "categories": [ { "id": "tutorial-transcript", "title": "Tutorial Transcript", "subcategories": [ { "id": "tutorial-getting-started", "title": "Getting started", "methods": [ { "id": "tutorial-ch01", "title": "Introduction", "keywords": [ "tutorial", "transcript", "video", "youtube", "introduction" ], "summary": "Walkthrough of Aurora (MouseMorph): a free, batch-oriented pipeline that unifies rigid alignment, elastic registration, 3D segmentation, and landmarking so large morphometric image datasets stay reproducible from raw scans to structured outputs.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Biological imaging gives us extraordinary views inside living systems. But the real challenge is turning those images into structured meaningful data.", "startSeconds": 0.56 }, { "text": "If you've ever worked with 3D biological or medical images, you know how quickly the workflow becomes fragmented.", "startSeconds": 15.6 }, { "text": "Cleaning files, aligning samples, running processes, creating annotations, organizing outputs, and repeating the same steps across dozens or hundreds of specimens.", "startSeconds": 23.8 }, { "text": "Along the way, data sets can become messy, naming conventions drift, and different tools or scripts are often needed just to move from one step to the next. Today, I want to present a free-to-use platform that was developed to bring many of these steps together into a single", "startSeconds": 37.56 }, { "text": "automated pipeline. Operations such as rigid alignment, elastic registration, 3D segmentation, and landmarking can be run in batch across the entirety of your data sets with sensible defaults and semi-automated methods that reduce the need for high manual tuning. The goal is to make complex image processing", "startSeconds": 58.12 }, { "text": "workflows easier to run in a reproducible way, especially when working with large data sets. While the main purpose of these tools are to be used in morphometric research, the pipeline can also support other applications such as data set preparation, volumetric measurements, or", "startSeconds": 82.12 }, { "text": "preprocessing for machine learning workflows. My name is Alejandro Gutierrez. I am a biomedical imaging specialist at the Alberta Children's Hospital Research Institute, and I'm the primary developer of this software. In this tutorial, I'll walk through how to install the platform and run a complete", "startSeconds": 102.56 }, { "text": "processing pipeline from raw image to structured outputs. So, to set it up, first you go to the Helgason Lab.ca website.", "startSeconds": 121.88 } ], "algorithm": { "text": "Biological imaging gives us extraordinary views inside living systems. But the real challenge is turning those images into structured meaningful data. If you've ever worked with 3D biological or medical images, you know how quickly the workflow becomes fragmented. Cleaning files, aligning samples, running processes, creating annotations, organizing outputs, and repeating the same steps across dozens or hundreds of specimens. Along the way, data sets can become messy, naming conventions drift, and different tools or scripts are often needed just to move from one step to the next. Today, I want to present a free-to-use platform that was developed to bring many of these steps together into a single automated pipeline. Operations such as rigid alignment, elastic registration, 3D segmentation, and landmarking can be run in batch across the entirety of your data sets with sensible defaults and semi-automated methods that reduce the need for high manual tuning. The goal is to make complex image processing workflows easier to run in a reproducible way, especially when working with large data sets. While the main purpose of these tools are to be used in morphometric research, the pipeline can also support other applications such as data set preparation, volumetric measurements, or preprocessing for machine learning workflows. My name is Alejandro Gutierrez. I am a biomedical imaging specialist at the Alberta Children's Hospital Research Institute, and I'm the primary developer of this software. In this tutorial, I'll walk through how to install the platform and run a complete processing pipeline from raw image to structured outputs. So, to set it up, first you go to the Helgason Lab.ca website.", "math": "", "diagram": "" }, "related": [ "tutorial-ch02" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 0, "chapterStart": "00:00" }, "images": [] }, { "id": "tutorial-ch02", "title": "Setting up an account", "keywords": [ "tutorial", "transcript", "video", "youtube", "setting up an account" ], "summary": "Create a Hallgrímsson Lab account, confirm your email, sign in (optional 2FA), then open the MouseMorph web application from the tools area.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Here, you will see in the top right, there's a login button. You go there. And if you don't have an account, you can go and click sign up, fill in your information, and then check your email to confirm.", "startSeconds": 137.48 }, { "text": "Uh make sure you check on the spam folder. Once I approve you, you can come back here, put in your credentials, sign in.", "startSeconds": 153.72 }, { "text": "You will see there's an optional two-factor authentication. I have it set up for myself, but you can decide if you want it or not.", "startSeconds": 171.32 }, { "text": "Once you're in, your name will show up on the top right, and you will see these three new options and in the web applications area.", "startSeconds": 187.4 }, { "text": "The toolkit that I'm going to show you is called Must Morph now. You click on that.", "startSeconds": 196.72 } ], "algorithm": { "text": "Here, you will see in the top right, there's a login button. You go there. And if you don't have an account, you can go and click sign up, fill in your information, and then check your email to confirm. Uh make sure you check on the spam folder. Once I approve you, you can come back here, put in your credentials, sign in. You will see there's an optional two-factor authentication. I have it set up for myself, but you can decide if you want it or not. Once you're in, your name will show up on the top right, and you will see these three new options and in the web applications area. The toolkit that I'm going to show you is called Must Morph now. You click on that.", "math": "", "diagram": "" }, "related": [ "tutorial-ch01", "tutorial-ch03" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 136, "chapterStart": "02:16" }, "images": [] }, { "id": "tutorial-ch03", "title": "Download and install the software", "keywords": [ "tutorial", "transcript", "video", "youtube", "download and install the software" ], "summary": "Download the OS-specific installer from the site, run the setup wizard, launch the MouseMorph app so its local terminal starts, then refresh the website to connect to the running client.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "You can hide the menu, and you will see that there will be a button here to download the software.", "startSeconds": 205.0 }, { "text": "Depending on what operating system you're in, you're going to see a different button appear here.", "startSeconds": 216.56 }, { "text": "So, then you can go ahead and click to download. It's downloading now. Once it's downloaded, you can go ahead and click on it.", "startSeconds": 222.52 }, { "text": "The wizard will open. You can hit next, next, install. I'm going to skip this part since I have it already installed. Once the installation successfully finished, you can go to start, search for MouseMorph", "startSeconds": 232.6 }, { "text": "app and you'll see it there. You can just go ahead and click on it. What you'll see is that a terminal will open and a bunch of information will be shown here. So, what we do is we refresh the website. We can now go ahead and hide", "startSeconds": 247.32 }, { "text": "this top part by clicking on this arrow.", "startSeconds": 261.84 } ], "algorithm": { "text": "You can hide the menu, and you will see that there will be a button here to download the software. Depending on what operating system you're in, you're going to see a different button appear here. So, then you can go ahead and click to download. It's downloading now. Once it's downloaded, you can go ahead and click on it. The wizard will open. You can hit next, next, install. I'm going to skip this part since I have it already installed. Once the installation successfully finished, you can go to start, search for MouseMorph app and you'll see it there. You can just go ahead and click on it. What you'll see is that a terminal will open and a bunch of information will be shown here. So, what we do is we refresh the website. We can now go ahead and hide this top part by clicking on this arrow.", "math": "", "diagram": "" }, "related": [ "tutorial-ch02", "tutorial-ch04" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 205, "chapterStart": "03:25" }, "images": [] }, { "id": "tutorial-ch04", "title": "File exploration and bundle detection (compatible formats)", "keywords": [ "tutorial", "transcript", "video", "youtube", "file exploration and bundle detection (compatible formats)" ], "summary": "Use the left-hand file explorer to browse your machine; Aurora auto-detects image bundles (TIFF folders, NIfTI, DICOM, Scanco AIM, multi-page TIFFs) and shows per-bundle summary info.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Uh first time you open it, it might take a while for uh the left side to load. Here, you can navigate some of the main paths in your computer.", "startSeconds": 264.36 }, { "text": "Uh for example, here I am in my E drive. So, I just clicked on the E drive and then went to my folder called Embryos Test video.", "startSeconds": 273.16 }, { "text": "So, for example, if I go to the home folder, it finds a bunch of files and documents. However, there's no bundles, which it shows it up here. What the bundles are is this is automatically detecting if in such folder there are any files or folders that look like medical images. In this case, it found", "startSeconds": 284.6 }, { "text": "that I have two TIFF folders and then one NIFTI image. Beyond these two file types, it can also accept DICOMs and the ScanCo machine AIM file as well as multi-page TIFFs. So, now the bundles will show up here and they will show", "startSeconds": 305.6 }, { "text": "some summary information of the images.", "startSeconds": 324.44 } ], "algorithm": { "text": "Uh first time you open it, it might take a while for uh the left side to load. Here, you can navigate some of the main paths in your computer. Uh for example, here I am in my E drive. So, I just clicked on the E drive and then went to my folder called Embryos Test video. So, for example, if I go to the home folder, it finds a bunch of files and documents. However, there's no bundles, which it shows it up here. What the bundles are is this is automatically detecting if in such folder there are any files or folders that look like medical images. In this case, it found that I have two TIFF folders and then one NIFTI image. Beyond these two file types, it can also accept DICOMs and the ScanCo machine AIM file as well as multi-page TIFFs. So, now the bundles will show up here and they will show some summary information of the images.", "math": "", "diagram": "" }, "related": [ "tutorial-ch03", "tutorial-ch05" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 264, "chapterStart": "04:24" }, "images": [] }, { "id": "tutorial-ch05", "title": "Extract all scans", "keywords": [ "tutorial", "transcript", "video", "youtube", "extract all scans" ], "summary": "Run Extract all scans to unpack bundles into viewable volumes—watching the terminal for voxel-size and isotropy handling—then use cyan/yellow status icons and the processing tools panel on the right.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "If we click on them, it'll say that to visualize the scan, we need to actually click on extract all scans, which is this first button that we see up here.", "startSeconds": 327.28 }, { "text": "So, we can go ahead and click on extract all scans. As we wait for it to complete extracting, I I to show you what's happening behind the scenes. So, if we click on the terminal, we can see here that a a bunch of things", "startSeconds": 339.12 }, { "text": "are being printed. If there's any issues you encounter with the software, please copy anything that got printed and send it to us. One thing that you can observe from this back-end functions running here is that we are fetching voxel sizes from the", "startSeconds": 352.76 }, { "text": "files and the headers. And we are determining if the voxels are isotropic. So, sometimes when we have MRI scans, you often find that the interslice distance is higher.", "startSeconds": 373.76 }, { "text": "So, that anisotropy will be calculated here. And then we interpolate using a registration method so that we can get isotropic images in the end. The speed is going to be determined by how powerful your computer is and the amount of tools you can use will be determined by how much", "startSeconds": 388.08 }, { "text": "RAM memory you have available. We suggest having at least 64 GB of RAM, but anything above that will be highly beneficial, especially for a larger scans. Once it is done, the page will refresh automatically.", "startSeconds": 411.8 }, { "text": "You will see that the images got extracted successfully because the icons here on the left will show in cyan.", "startSeconds": 428.6 }, { "text": "Otherwise, if they were partially extracted due to an error, you will see them in color yellow. So, we can go ahead now and click on the first one.", "startSeconds": 437.16 }, { "text": "Okay, so now we're going to focus on the right side of the interface. Um the first thing we see is a block here with a lot of options. These are all the processing steps that we can run on our images. Most of them work on batch level, so all the images others uh can be set up to work only on specific", "startSeconds": 445.64 }, { "text": "selected images.", "startSeconds": 465.84 } ], "algorithm": { "text": "If we click on them, it'll say that to visualize the scan, we need to actually click on extract all scans, which is this first button that we see up here. So, we can go ahead and click on extract all scans. As we wait for it to complete extracting, I I to show you what's happening behind the scenes. So, if we click on the terminal, we can see here that a a bunch of things are being printed. If there's any issues you encounter with the software, please copy anything that got printed and send it to us. One thing that you can observe from this back-end functions running here is that we are fetching voxel sizes from the files and the headers. And we are determining if the voxels are isotropic. So, sometimes when we have MRI scans, you often find that the interslice distance is higher. So, that anisotropy will be calculated here. And then we interpolate using a registration method so that we can get isotropic images in the end. The speed is going to be determined by how powerful your computer is and the amount of tools you can use will be determined by how much RAM memory you have available. We suggest having at least 64 GB of RAM, but anything above that will be highly beneficial, especially for a larger scans. Once it is done, the page will refresh automatically. You will see that the images got extracted successfully because the icons here on the left will show in cyan. Otherwise, if they were partially extracted due to an error, you will see them in color yellow. So, we can go ahead now and click on the first one. Okay, so now we're going to focus on the right side of the interface. Um the first thing we see is a block here with a lot of options. These are all the processing steps that we can run on our images. Most of them work on batch level, so all the images others uh can be set up to work only on specific selected images.", "math": "", "diagram": "" }, "related": [ "tutorial-ch04", "tutorial-ch06" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 327, "chapterStart": "05:27" }, "images": [] }, { "id": "tutorial-ch06", "title": "Reference scan selection importance", "keywords": [ "tutorial", "transcript", "video", "youtube", "reference scan selection importance" ], "summary": "Pick a clean, average-looking reference scan carefully: later batch tools largely copy the reference’s rotation, crop, intensity, and other edits onto the rest of the dataset.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Then we have a reference scan block. Here you're supposed to choose a scan that looks like your most clean and average-looking scan. Selecting the reference scan is a non-trivial process because a lot of the different tools we have up here will rely on this reference scan to", "startSeconds": 468.24 }, { "text": "make their job. Basically, a lot of the steps will try to imitate whatever you already did to the reference scan, such as rotating it, scaling it, cropping it, changing the intensity levels, etc. And then in the", "startSeconds": 488.84 } ], "algorithm": { "text": "Then we have a reference scan block. Here you're supposed to choose a scan that looks like your most clean and average-looking scan. Selecting the reference scan is a non-trivial process because a lot of the different tools we have up here will rely on this reference scan to make their job. Basically, a lot of the steps will try to imitate whatever you already did to the reference scan, such as rotating it, scaling it, cropping it, changing the intensity levels, etc. And then in the", "math": "", "diagram": "" }, "related": [ "tutorial-ch05", "tutorial-ch07" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 467, "chapterStart": "07:47" }, "images": [] }, { "id": "tutorial-ch07", "title": "Preview block overview", "keywords": [ "tutorial", "transcript", "video", "youtube", "preview block overview" ], "summary": "The preview block shows edit history for the current scan (starting from the raw extract) and stacks each new processing step as you work.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "bottom, we can see the preview block. In the preview block, we have different things. First, on the top left, we have some summary information of the image with whichever edits we will make to the image.", "startSeconds": 502.84 }, { "text": "Here we, as we just extracted it, we only see the raw edit. But then, every time we do something to the image, we will see new edits stack on here. On", "startSeconds": 516.72 } ], "algorithm": { "text": "bottom, we can see the preview block. In the preview block, we have different things. First, on the top left, we have some summary information of the image with whichever edits we will make to the image. Here we, as we just extracted it, we only see the raw edit. But then, every time we do something to the image, we will see new edits stack on here. On", "math": "", "diagram": "" }, "related": [ "tutorial-ch06", "tutorial-ch08" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 501, "chapterStart": "08:21" }, "images": [] }, { "id": "tutorial-ch08", "title": "Threshold adjustment and mesh generation", "keywords": [ "tutorial", "transcript", "video", "youtube", "threshold adjustment and mesh generation" ], "summary": "Tune the marching-cubes mesh with the histogram’s yellow threshold bar, refresh to inspect the surface, then save so the chosen threshold sticks with the scan.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "the right side, we see the mesh that got automatically generated by using a threshold, and then some options to process the image that rely on the mesh visualization to execute. Below, you will see a histogram. The histogram represents the distribution of intensity", "startSeconds": 528.92 }, { "text": "values of the image. And then you will see a vertical yellow bar. This vertical yellow bar represents the threshold that is currently used to generate this mesh.", "startSeconds": 547.04 }, { "text": "So, if we click on threshold mode up here, now we will see that this is the actual threshold that is generating this mesh using uh marching cubes algorithm, if you're curious.", "startSeconds": 558.64 }, { "text": "So, now we can change the threshold by sliding this yellow bar here. We will try to find a threshold that correctly encapsulates all the different anatomy without introducing a lot of noise, as we can see when we get too close to the", "startSeconds": 571.48 }, { "text": "background here. So, this should be good. Before you save the value, if you want to check how that looks on the mesh, you can go in the top right here and click refresh. As you can see, now we see a different mesh. Now, there's less holes around it cuz the previous threshold was", "startSeconds": 589.84 }, { "text": "too restrictive is when it looks better. If you want to see the mesh in up closer, you can go and toggle full screen down here.", "startSeconds": 611.2 }, { "text": "And now we can see the full mesh in full screen. If you are happy with this threshold, then you can just hit the save icon.", "startSeconds": 622.12 }, { "text": "The website reloads, and here we have it. The scan now saved the threshold.", "startSeconds": 635.24 } ], "algorithm": { "text": "the right side, we see the mesh that got automatically generated by using a threshold, and then some options to process the image that rely on the mesh visualization to execute. Below, you will see a histogram. The histogram represents the distribution of intensity values of the image. And then you will see a vertical yellow bar. This vertical yellow bar represents the threshold that is currently used to generate this mesh. So, if we click on threshold mode up here, now we will see that this is the actual threshold that is generating this mesh using uh marching cubes algorithm, if you're curious. So, now we can change the threshold by sliding this yellow bar here. We will try to find a threshold that correctly encapsulates all the different anatomy without introducing a lot of noise, as we can see when we get too close to the background here. So, this should be good. Before you save the value, if you want to check how that looks on the mesh, you can go in the top right here and click refresh. As you can see, now we see a different mesh. Now, there's less holes around it cuz the previous threshold was too restrictive is when it looks better. If you want to see the mesh in up closer, you can go and toggle full screen down here. And now we can see the full mesh in full screen. If you are happy with this threshold, then you can just hit the save icon. The website reloads, and here we have it. The scan now saved the threshold.", "math": "", "diagram": "" }, "related": [ "tutorial-ch07", "tutorial-ch09" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 528, "chapterStart": "08:48" }, "images": [] }, { "id": "tutorial-ch09", "title": "Voxel image viewer (adjusting intensity value preview)", "keywords": [ "tutorial", "transcript", "video", "youtube", "voxel image viewer (adjusting intensity value preview)" ], "summary": "Navigate sagittal, axial, and coronal slice views with crosshairs to read voxel intensities, and use the histogram display range (with percentile labels) to preview contrast before normalization.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "And last but not least, we have at the bottom here of the preview the intensity values. So, we have three windows for the different sagittal, axial, and coronal views.", "startSeconds": 640.16 }, { "text": "If we click and drag on any of the three windows, we can slide around with the crosshairs to see all the different slices.", "startSeconds": 652.16 }, { "text": "At the intersection of the crosshairs, it will calculate what the unit for that voxel is, and it will show you that on the top left here.", "startSeconds": 670.2 }, { "text": "So, for example, we see that this background area here has values of around 11,000.", "startSeconds": 682.12 }, { "text": "So, what you can do now is to visualize it a little bit cleaner. You can go up here to the histogram and drag the slider on the bottom until we reach that 11,000.", "startSeconds": 691.76 }, { "text": "Now you see that the background is completely black. And you can do the same for the foreground. So the top slider you can bring it down.", "startSeconds": 707.92 }, { "text": "And now the range of intensities will be more evenly distributed from black to white. Anything that lies outside of the range that you visualize here will just go black.", "startSeconds": 721.52 }, { "text": "So bear that in mind. Finally, something that you can see here in this histogram is that the sliders have a percentage on top of them.", "startSeconds": 733.12 }, { "text": "This percentage represents the percentile given by the distribution of intensity values at that specific value.", "startSeconds": 741.36 }, { "text": "This will become helpful later when we want to normalize the values. Okay,", "startSeconds": 749.32 } ], "algorithm": { "text": "And last but not least, we have at the bottom here of the preview the intensity values. So, we have three windows for the different sagittal, axial, and coronal views. If we click and drag on any of the three windows, we can slide around with the crosshairs to see all the different slices. At the intersection of the crosshairs, it will calculate what the unit for that voxel is, and it will show you that on the top left here. So, for example, we see that this background area here has values of around 11,000. So, what you can do now is to visualize it a little bit cleaner. You can go up here to the histogram and drag the slider on the bottom until we reach that 11,000. Now you see that the background is completely black. And you can do the same for the foreground. So the top slider you can bring it down. And now the range of intensities will be more evenly distributed from black to white. Anything that lies outside of the range that you visualize here will just go black. So bear that in mind. Finally, something that you can see here in this histogram is that the sliders have a percentage on top of them. This percentage represents the percentile given by the distribution of intensity values at that specific value. This will become helpful later when we want to normalize the values. Okay,", "math": "", "diagram": "" }, "related": [ "tutorial-ch08", "tutorial-ch10" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 640, "chapterStart": "10:40" }, "images": [] }, { "id": "tutorial-ch10", "title": "Important quality checks (how to mark scans as faulty | use grid view)", "keywords": [ "tutorial", "transcript", "video", "youtube", "important quality checks (how to mark scans as faulty | use grid view)" ], "summary": "QC the cohort: flag bad scans as faulty (so batch tools skip them), use grid view to compare projections/meshes/thresholds, and fix per-scan thresholds or hide faulty items as needed.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "first thing I'm going to do is I'm going to check the images themselves. I'm going to see that the quality is good enough. If any of them show any signs of artifacts or undesired anatomy, then I can come and mark them", "startSeconds": 753.72 }, { "text": "as faulty. To do that you go on the right side of the title and there's this button here called report as faulty.", "startSeconds": 769.36 }, { "text": "It'll prompt you to confirm. You hit okay and then you'll see it reloads and now it shows you this icon on the right side of of the scan saying this scan has been marked as faulty.", "startSeconds": 777.36 }, { "text": "What this means is that for every single processing step that you can run, it will actively ignore this particular scan without actually deleting it if you want to revisit it at some point in the future.", "startSeconds": 789.84 }, { "text": "Something else you can do here is hide all of them. So, in the working directory on the top, there's a button that says hide faulty scans.", "startSeconds": 806.4 }, { "text": "You click that, it'll hide it. If you change your mind, you can come back to the scan and click on this again, and it will say, \"Do you want to unflag this scan?\" You can say okay.", "startSeconds": 813.88 }, { "text": "It will reload. And now we're back to where we started. Part of this initial revision of all the scans will have to include checking the automatic thresholds that they got assigned for their own meshes.", "startSeconds": 825.44 }, { "text": "We already went through the main image here, but you have two options. You either click on all the subsequent scans one by one to check the threshold, which would be the highest quality way to test this.", "startSeconds": 839.84 }, { "text": "Or, you can go ahead and click on grid view, which is in the top right corner of the working directory block. So, now that they loaded, we can see them all side by side.", "startSeconds": 852.68 }, { "text": "This grid view shows you several things. First, it shows you a projection of one of the axis, essentially flattening the image as if it was an X-ray. Then, you see the other two different views at their center slice. And then, you see a", "startSeconds": 862.96 }, { "text": "preview of the marching cubes 3D mesh. But, this is all static. You cannot rotate the mesh or anything like that.", "startSeconds": 877.08 }, { "text": "And it's a low quality mesh. So, don't rely too much on it. You can also see a preview of the histogram for the image and a preview of the yellow vertical bar representing the threshold. So, for example here, we can see that this scan on the right side has a very decent looking", "startSeconds": 882.64 }, { "text": "threshold, but the one on the bottom here looks like very destructive. One way to quickly inspect this is we go on the right side here, and we have an option that says threshold selection.", "startSeconds": 902.92 }, { "text": "We toggle that on, and we actually see a preview of the thresholds on these bottom images, and we can adjust individual thresholds. So, for example, the bottom image that got a wrong one, we click once anywhere in the histogram, and now the yellow vertical bar follows my mouse. Okay, so now I can", "startSeconds": 915.4 }, { "text": "test by moving it a little closer to the background peak. So, I click again, and that shows me how the threshold moved.", "startSeconds": 935.44 }, { "text": "Again, this is applying it to the whole image, but it's actually only true for this bottom row. So, I'm going to adjust it a little more. Somewhere in there looks good. So, now I'm going to go ahead on the right side and hit save", "startSeconds": 943.84 }, { "text": "threshold values, which will reload it. Now, if we go and click on grid view again, it will reprocess whichever images had some changes, but automatically load back any previews that had no changes made to them. Here we go, we're back here, and now we see that the preview of the mesh that it", "startSeconds": 957.36 }, { "text": "generated with marching cubes looks way better. You shouldn't still fully rely on this. This is decent, and I can work with this, but if you are very interested in having a very accurate mesh, you should definitely go and click on the image and use the different tools that we have on the preview. One thing", "startSeconds": 974.72 }, { "text": "that we can notice here, for example, is that this bottom left scan it kind of looks like the contrast at the center of the image is washed out compared to the outer edges. This could have probably been due to different factors during scanning, during the staining of the sample, etc. If we feel like this is", "startSeconds": 991.8 }, { "text": "something we don't want to work with, we can also flag it as faulty in the grid view by clicking on the top right icon.", "startSeconds": 1011.4 }, { "text": "For now, I'm going to keep it because I'm only interested in the outer mesh for the analysis I'm going to run. Other things you can do in this view are zooming in into the different quadrants.", "startSeconds": 1018.84 }, { "text": "So, for example, if I only want to see the 3D mesh preview, I can hit on this one and it'll just zoom in.", "startSeconds": 1032.64 } ], "algorithm": { "text": "first thing I'm going to do is I'm going to check the images themselves. I'm going to see that the quality is good enough. If any of them show any signs of artifacts or undesired anatomy, then I can come and mark them as faulty. To do that you go on the right side of the title and there's this button here called report as faulty. It'll prompt you to confirm. You hit okay and then you'll see it reloads and now it shows you this icon on the right side of of the scan saying this scan has been marked as faulty. What this means is that for every single processing step that you can run, it will actively ignore this particular scan without actually deleting it if you want to revisit it at some point in the future. Something else you can do here is hide all of them. So, in the working directory on the top, there's a button that says hide faulty scans. You click that, it'll hide it. If you change your mind, you can come back to the scan and click on this again, and it will say, \"Do you want to unflag this scan?\" You can say okay. It will reload. And now we're back to where we started. Part of this initial revision of all the scans will have to include checking the automatic thresholds that they got assigned for their own meshes. We already went through the main image here, but you have two options. You either click on all the subsequent scans one by one to check the threshold, which would be the highest quality way to test this. Or, you can go ahead and click on grid view, which is in the top right corner of the working directory block. So, now that they loaded, we can see them all side by side. This grid view shows you several things. First, it shows you a projection of one of the axis, essentially flattening the image as if it was an X-ray. Then, you see the other two different views at their center slice. And then, you see a preview of the marching cubes 3D mesh. But, this is all static. You cannot rotate the mesh or anything like that. And it's a low quality mesh. So, don't rely too much on it. You can also see a preview of the histogram for the image and a preview of the yellow vertical bar representing the threshold. So, for example here, we can see that this scan on the right side has a very decent looking threshold, but the one on the bottom here looks like very destructive. One way to quickly inspect this is we go on the right side here, and we have an option that says threshold selection. We toggle that on, and we actually see a preview of the thresholds on these bottom images, and we can adjust individual thresholds. So, for example, the bottom image that got a wrong one, we click once anywhere in the histogram, and now the yellow vertical bar follows my mouse. Okay, so now I can test by moving it a little closer to the background peak. So, I click again, and that shows me how the threshold moved. Again, this is applying it to the whole image, but it's actually only true for this bottom row. So, I'm going to adjust it a little more. Somewhere in there looks good. So, now I'm going to go ahead on the right side and hit save threshold values, which will reload it. Now, if we go and click on grid view again, it will reprocess whichever images had some changes, but automatically load back any previews that had no changes made to them. Here we go, we're back here, and now we see that the preview of the mesh that it generated with marching cubes looks way better. You shouldn't still fully rely on this. This is decent, and I can work with this, but if you are very interested in having a very accurate mesh, you should definitely go and click on the image and use the different tools that we have on the preview. One thing that we can notice here, for example, is that this bottom left scan it kind of looks like the contrast at the center of the image is washed out compared to the outer edges. This could have probably been due to different factors during scanning, during the staining of the sample, etc. If we feel like this is something we don't want to work with, we can also flag it as faulty in the grid view by clicking on the top right icon. For now, I'm going to keep it because I'm only interested in the outer mesh for the analysis I'm going to run. Other things you can do in this view are zooming in into the different quadrants. So, for example, if I only want to see the 3D mesh preview, I can hit on this one and it'll just zoom in.", "math": "", "diagram": "" }, "related": [ "tutorial-ch09", "tutorial-ch11" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 753, "chapterStart": "12:33" }, "images": [] }, { "id": "tutorial-ch11", "title": "Choosing a reference after the quality checks", "keywords": [ "tutorial", "transcript", "video", "youtube", "choosing a reference after the quality checks" ], "summary": "After QC, choose the most representative scan (good contrast in regions of interest) from the reference dropdown; a ruler icon marks it as the active reference.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Finally, what I can see from here is that I actually like this first image to be the reference. It looks pretty average. It looks like there's a decent amount of contrast, especially in the areas I'm interested in, which is the", "startSeconds": 1047.32 }, { "text": "head and the face here. So, then I can go ahead and click on it. Now, since I like this one, I can just go and select it from the drop-down menu here in the reference scan block.", "startSeconds": 1060.52 }, { "text": "And you will see that it adds this ruler icon on the top right here. So, now the", "startSeconds": 1072.76 } ], "algorithm": { "text": "Finally, what I can see from here is that I actually like this first image to be the reference. It looks pretty average. It looks like there's a decent amount of contrast, especially in the areas I'm interested in, which is the head and the face here. So, then I can go ahead and click on it. Now, since I like this one, I can just go and select it from the drop-down menu here in the reference scan block. And you will see that it adds this ruler icon on the top right here. So, now the", "math": "", "diagram": "" }, "related": [ "tutorial-ch10" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1047, "chapterStart": "17:27" }, "images": [] } ] }, { "id": "tutorial-processing-the-reference-scan", "title": "Processing the reference scan", "methods": [ { "id": "tutorial-ch12", "title": "Set 3D Rotation manually", "keywords": [ "tutorial", "transcript", "video", "youtube", "set 3d rotation manually" ], "summary": "Manually rotate the reference in mesh rotate mode using axis guides and the 3×3 matrix, then apply (usually to this scan only) so both mesh and voxels share a standardized orientation.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "first thing we want to do is we want to realign this scan to fit our standardized criteria. To do that, we first should apply a rotation. That will be by going here in the mesh to rotate mode. This should only be necessary for your reference scan, unless there's something special that needs to be done", "startSeconds": 1078.72 }, { "text": "for any others. You can see now that we have three sliders up here. Each of the sliders rotates a different axis.", "startSeconds": 1098.4 }, { "text": "On the left, we see a window with a 3x3 matrix. This matrix represents the rotation we want to apply. We can go ahead and go into full screen view to make it easier for us to run this step. Within this view, we can start moving the sliders around,", "startSeconds": 1106.96 }, { "text": "rotating the mesh. And as we do that, we see that the rotation matrix starts to change.", "startSeconds": 1125.56 }, { "text": "If we want to reset to the original position, we can just click reset here and we go back to the original orientation. So, what I like to do to get a very good rotation here is I go first down here and hover over my three different views,", "startSeconds": 1134.24 }, { "text": "which is to align it to Z, Y, and X. The first one I do, I like to do Z, which is the front facing.", "startSeconds": 1150.76 }, { "text": "As you can see, uh in this one, we get a diagram that shows us that the specimen should be facing us.", "startSeconds": 1159.88 }, { "text": "And then the blue circle will be shown around it, right? So, we click this and it automatically reorients us to that position. Here, what I do first is I quickly try to make the specimen be aligned as it suggested.", "startSeconds": 1168.04 }, { "text": "I still don't try to make it perfect, just very roughly looking at me. Okay. Now, I go to the Y view. Now, in the Y view, you can see it's a top-down view and it shows us that we should be looking at the specimen facing down. So, now we focus on the green specifically until we're happy with it. Then, we go", "startSeconds": 1184.2 }, { "text": "to the X view, which tells us that the specimen should be looking to the left. Here, I'm going to focus then only on the red ring and I'm going to align it until I see a position where I would", "startSeconds": 1205.56 }, { "text": "like in the future to crop the image. What do I mean with this? Is if I were to crop the image from the bottom, would it include part of the bottom of the mouth here that I want to keep or not?", "startSeconds": 1217.08 }, { "text": "So, for example, if I did this and then I crop the bottom, I would likely have to crop part of the mouth to get higher on the neck here. So, I like this one.", "startSeconds": 1229.88 }, { "text": "And once I'm happy with this, I go back to Z, so the original, and I fix the final Z alignment. Okay, this looks pretty symmetrical to me.", "startSeconds": 1240.04 }, { "text": "Now, I'm being more picky about it, fine-tuning it. I go to Y again. Y looks pretty good.", "startSeconds": 1250.8 }, { "text": "And then X again, and X looks pretty good. Okay, so now I'm happy with it. Now, we can apply this rotation. You can either apply it to this scan alone or to all the scans. If you apply it to all the scans, essentially what you're saying is that all of the scans were already in the", "startSeconds": 1259.36 }, { "text": "same position. That is not the case here as you saw. Every scan is in a different position.", "startSeconds": 1276.16 }, { "text": "So, I'm going to only apply it to this scan. You shouldn't need to touch the interpolation method. It will show you that if you change it to nearest neighbor, that is only necessary for binary images. That is if you were applying this rotation for a", "startSeconds": 1281.48 }, { "text": "segmentation mask, for example. So, we keep it as linear, and then we just hit apply the scan, and then we just wait. Okay, so now that it finished, we can go ahead and click on it. One thing you can notice is that now on the same bundle, now you have this", "startSeconds": 1296.24 }, { "text": "little icon here that says 3D rotated. This will be indicating each edit that you do to any of the images. This literally means that a new file got created that has the 3D rotation integrated on it.", "startSeconds": 1311.72 }, { "text": "As you see now, the mesh is nicely aligned, but also the voxel image down here is very nicely aligned.", "startSeconds": 1324.2 } ], "algorithm": { "text": "first thing we want to do is we want to realign this scan to fit our standardized criteria. To do that, we first should apply a rotation. That will be by going here in the mesh to rotate mode. This should only be necessary for your reference scan, unless there's something special that needs to be done for any others. You can see now that we have three sliders up here. Each of the sliders rotates a different axis. On the left, we see a window with a 3x3 matrix. This matrix represents the rotation we want to apply. We can go ahead and go into full screen view to make it easier for us to run this step. Within this view, we can start moving the sliders around, rotating the mesh. And as we do that, we see that the rotation matrix starts to change. If we want to reset to the original position, we can just click reset here and we go back to the original orientation. So, what I like to do to get a very good rotation here is I go first down here and hover over my three different views, which is to align it to Z, Y, and X. The first one I do, I like to do Z, which is the front facing. As you can see, uh in this one, we get a diagram that shows us that the specimen should be facing us. And then the blue circle will be shown around it, right? So, we click this and it automatically reorients us to that position. Here, what I do first is I quickly try to make the specimen be aligned as it suggested. I still don't try to make it perfect, just very roughly looking at me. Okay. Now, I go to the Y view. Now, in the Y view, you can see it's a top-down view and it shows us that we should be looking at the specimen facing down. So, now we focus on the green specifically until we're happy with it. Then, we go to the X view, which tells us that the specimen should be looking to the left. Here, I'm going to focus then only on the red ring and I'm going to align it until I see a position where I would like in the future to crop the image. What do I mean with this? Is if I were to crop the image from the bottom, would it include part of the bottom of the mouth here that I want to keep or not? So, for example, if I did this and then I crop the bottom, I would likely have to crop part of the mouth to get higher on the neck here. So, I like this one. And once I'm happy with this, I go back to Z, so the original, and I fix the final Z alignment. Okay, this looks pretty symmetrical to me. Now, I'm being more picky about it, fine-tuning it. I go to Y again. Y looks pretty good. And then X again, and X looks pretty good. Okay, so now I'm happy with it. Now, we can apply this rotation. You can either apply it to this scan alone or to all the scans. If you apply it to all the scans, essentially what you're saying is that all of the scans were already in the same position. That is not the case here as you saw. Every scan is in a different position. So, I'm going to only apply it to this scan. You shouldn't need to touch the interpolation method. It will show you that if you change it to nearest neighbor, that is only necessary for binary images. That is if you were applying this rotation for a segmentation mask, for example. So, we keep it as linear, and then we just hit apply the scan, and then we just wait. Okay, so now that it finished, we can go ahead and click on it. One thing you can notice is that now on the same bundle, now you have this little icon here that says 3D rotated. This will be indicating each edit that you do to any of the images. This literally means that a new file got created that has the 3D rotation integrated on it. As you see now, the mesh is nicely aligned, but also the voxel image down here is very nicely aligned.", "math": "", "diagram": "" }, "related": [ "tutorial-ch13" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1078, "chapterStart": "17:58" }, "images": [] }, { "id": "tutorial-ch13", "title": "Storage awareness", "keywords": [ "tutorial", "transcript", "video", "youtube", "storage awareness" ], "summary": "Each edit writes a new file—storage grows with cohort size and pipeline depth—so monitor disk usage early on large projects.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "You have to be careful about a storage because every time you do an edit, a new file is created.", "startSeconds": 1334.92 }, { "text": "So, if you go here, you can see that we're occupying 11 GB out of 1.3 TB. You might say, \"Oh, this is plenty for what we need.\" But once you start running larger projects like 150, 200 images, then each step multiplies. Okay,", "startSeconds": 1342.16 } ], "algorithm": { "text": "You have to be careful about a storage because every time you do an edit, a new file is created. So, if you go here, you can see that we're occupying 11 GB out of 1.3 TB. You might say, \"Oh, this is plenty for what we need.\" But once you start running larger projects like 150, 200 images, then each step multiplies. Okay,", "math": "", "diagram": "" }, "related": [ "tutorial-ch12", "tutorial-ch14" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1334, "chapterStart": "22:14" }, "images": [] }, { "id": "tutorial-ch14", "title": "Set cropping manually", "keywords": [ "tutorial", "transcript", "video", "youtube", "set cropping manually" ], "summary": "Crop the reference in crop mode with axis sliders, mesh-overlap cues, optional auto-fit, and padding so subjects have room to align; apply and verify the new bounds.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "for the next step, we're going to do some croppings. So, if we look at the image right now, there's some extra space that we don't need. Maybe there's some part of the anatomy that we just don't want.", "startSeconds": 1360.52 }, { "text": "And for that, you come to the preview where the mesh is, and you'll see these icons on the right side. The third one says crop mode. We click on that.", "startSeconds": 1371.68 }, { "text": "What we see here is the actual volume containing the image representing is in this red box. And then, on the left side, we see a little window with some options.", "startSeconds": 1382.92 }, { "text": "So, let's go into full screen and walk through each of these. First of all, we have these sliders that change the cropping.", "startSeconds": 1394.24 }, { "text": "The way to think about this is when we move the red slider, the red lines will shrink, okay?", "startSeconds": 1402.2 }, { "text": "So, we move the red slider, red lines shrink. Something you can notice here is that every time the boundaries of the box cross the mesh, the parts of the mesh that are crossing the box will be painted in a different color. So, you see whatever is going to be inside and what's going to be outside of your", "startSeconds": 1411.64 }, { "text": "image. And you'll get a warning here that says how many vertices are being left out, just in case it's an accident and you actually didn't mean to remove anything from it. Uh, you'll also see from which voxel to which voxel on each axis you are going to be doing the cropping. We have an option here that is", "startSeconds": 1430.56 }, { "text": "glowing. And this button here that's called auto fit crop to mesh. When you click this, it will automatically give you the smallest bounding box that contains the mesh.", "startSeconds": 1449.04 }, { "text": "If your image is noisy and you have floating specs all around, this is not going to work very well because then it's just going to contain everything including those floating specs. Another option you have here is", "startSeconds": 1461.4 }, { "text": "show all scans. Um, it's not going to work right now because show all scans only works when all of your scans are the same exact size.", "startSeconds": 1473.84 }, { "text": "And what it does is it loads them all together and shows them to you on top of each other. This way you can crop to a bounding box that contains all of your scans combined. Now, since we are cropping the reference, but we need a room for all the other scans to align to this one,", "startSeconds": 1486.0 }, { "text": "then we need to add a padding. By default, the padding is 40 voxels. However, if you see a lot of size variation in your data, you probably will want this padding to be larger.", "startSeconds": 1505.84 }, { "text": "Since my embryos here are not that variable in size, I'm just going to use 40. This essentially adds 40 voxels on each side of each axis of the box. Okay, so here for example, I did the smallest bounding box, but now I do not want the bottom of the neck. So, I can go below here and align to the x-axis, and", "startSeconds": 1517.96 }, { "text": "then shrink the green line, as you can see here. I still want the mouth, so I'm just going to crop right around there. Okay, and now that we have that, we can probably get away with shrinking a", "startSeconds": 1542.44 }, { "text": "little bit more of the red. Perfect. So, what this is going to do is it's going to crop it first to this exact volume, then it will add padding with values that are assigned as background, which in this case will be zero. So, once you're ready and you like", "startSeconds": 1556.0 }, { "text": "this, we hit apply to scan, and we wait. Okay, so now we have the image cropped to that size. We can verify by going again into the crop mode, and now we see the red box, which has the same cropping that we established, plus the 40 voxels on each side. We can also see that the cropping got applied", "startSeconds": 1592.68 }, { "text": "to the actual voxel image. Okay, now", "startSeconds": 1613.44 } ], "algorithm": { "text": "for the next step, we're going to do some croppings. So, if we look at the image right now, there's some extra space that we don't need. Maybe there's some part of the anatomy that we just don't want. And for that, you come to the preview where the mesh is, and you'll see these icons on the right side. The third one says crop mode. We click on that. What we see here is the actual volume containing the image representing is in this red box. And then, on the left side, we see a little window with some options. So, let's go into full screen and walk through each of these. First of all, we have these sliders that change the cropping. The way to think about this is when we move the red slider, the red lines will shrink, okay? So, we move the red slider, red lines shrink. Something you can notice here is that every time the boundaries of the box cross the mesh, the parts of the mesh that are crossing the box will be painted in a different color. So, you see whatever is going to be inside and what's going to be outside of your image. And you'll get a warning here that says how many vertices are being left out, just in case it's an accident and you actually didn't mean to remove anything from it. Uh, you'll also see from which voxel to which voxel on each axis you are going to be doing the cropping. We have an option here that is glowing. And this button here that's called auto fit crop to mesh. When you click this, it will automatically give you the smallest bounding box that contains the mesh. If your image is noisy and you have floating specs all around, this is not going to work very well because then it's just going to contain everything including those floating specs. Another option you have here is show all scans. Um, it's not going to work right now because show all scans only works when all of your scans are the same exact size. And what it does is it loads them all together and shows them to you on top of each other. This way you can crop to a bounding box that contains all of your scans combined. Now, since we are cropping the reference, but we need a room for all the other scans to align to this one, then we need to add a padding. By default, the padding is 40 voxels. However, if you see a lot of size variation in your data, you probably will want this padding to be larger. Since my embryos here are not that variable in size, I'm just going to use 40. This essentially adds 40 voxels on each side of each axis of the box. Okay, so here for example, I did the smallest bounding box, but now I do not want the bottom of the neck. So, I can go below here and align to the x-axis, and then shrink the green line, as you can see here. I still want the mouth, so I'm just going to crop right around there. Okay, and now that we have that, we can probably get away with shrinking a little bit more of the red. Perfect. So, what this is going to do is it's going to crop it first to this exact volume, then it will add padding with values that are assigned as background, which in this case will be zero. So, once you're ready and you like this, we hit apply to scan, and we wait. Okay, so now we have the image cropped to that size. We can verify by going again into the crop mode, and now we see the red box, which has the same cropping that we established, plus the 40 voxels on each side. We can also see that the cropping got applied to the actual voxel image. Okay, now", "math": "", "diagram": "" }, "related": [ "tutorial-ch13", "tutorial-ch15" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1360, "chapterStart": "22:40" }, "images": [] }, { "id": "tutorial-ch15", "title": "Background offset correction manually", "keywords": [ "tutorial", "transcript", "video", "youtube", "background offset correction manually" ], "summary": "Set background intensity on the reference (from the crosshair or batch auto mode using the thresholded background mode) so background voxels go to zero and the histogram/threshold shift accordingly.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "that we have the rotation and the cropping and you can see them right here as the edits. You can also see them on the bundle. The next step for our reference image is usually to establish the background value. So, if we again click and drag our crosshair on the image here, we see that the values of", "startSeconds": 1616.04 }, { "text": "the background are around 12,000 as seen on the top left here. So, if we want to just establish the background value for this image, we go down on the bottom right here on the settings icon and we find there's an option here called set", "startSeconds": 1636.08 }, { "text": "background. By default, it will pick up that value that the crosshair is at and it will suggest it here as the background.", "startSeconds": 1653.6 }, { "text": "We can just go ahead and click apply. And we wait. Okay, now that it's done, we can see that it reloaded and we have the image with the background all set to zero.", "startSeconds": 1662.48 }, { "text": "In the histogram, we can see it because it's the largest peak over here. What happened is that all the intensity values got shifted to the left. So, the same number was subtracted from every single voxel and the threshold got adjusted in the same way. Another thing you can do", "startSeconds": 1674.8 }, { "text": "um is to set the background instead of manually just doing it for this one scan, you can go up here and there's this option here that says set background value.", "startSeconds": 1692.72 }, { "text": "Here you can as well just set one number, but for all the images together or you can set the background automatically by clicking this yellow button. The way the automatic mode works is using the same threshold you establish here, it will only take the values that are", "startSeconds": 1703.84 }, { "text": "belonging to the background according to this threshold and then it will calculate the mode, the most frequent number, and it will use that as the background value. By the way, everything is explained in detail if you go into here and then hover over the buttons, it", "startSeconds": 1722.8 }, { "text": "will just explain what everything does. Another step you can do from the", "startSeconds": 1740.64 } ], "algorithm": { "text": "that we have the rotation and the cropping and you can see them right here as the edits. You can also see them on the bundle. The next step for our reference image is usually to establish the background value. So, if we again click and drag our crosshair on the image here, we see that the values of the background are around 12,000 as seen on the top left here. So, if we want to just establish the background value for this image, we go down on the bottom right here on the settings icon and we find there's an option here called set background. By default, it will pick up that value that the crosshair is at and it will suggest it here as the background. We can just go ahead and click apply. And we wait. Okay, now that it's done, we can see that it reloaded and we have the image with the background all set to zero. In the histogram, we can see it because it's the largest peak over here. What happened is that all the intensity values got shifted to the left. So, the same number was subtracted from every single voxel and the threshold got adjusted in the same way. Another thing you can do um is to set the background instead of manually just doing it for this one scan, you can go up here and there's this option here that says set background value. Here you can as well just set one number, but for all the images together or you can set the background automatically by clicking this yellow button. The way the automatic mode works is using the same threshold you establish here, it will only take the values that are belonging to the background according to this threshold and then it will calculate the mode, the most frequent number, and it will use that as the background value. By the way, everything is explained in detail if you go into here and then hover over the buttons, it will just explain what everything does. Another step you can do from the", "math": "", "diagram": "" }, "related": [ "tutorial-ch14", "tutorial-ch16" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1615, "chapterStart": "26:55" }, "images": [] }, { "id": "tutorial-ch16", "title": "Set N4 bias correction (for MRI)", "keywords": [ "tutorial", "transcript", "video", "youtube", "set n4 bias correction (for mri)" ], "summary": "Use N4 bias correction mainly for MRI when slow intensity gradients make one side of the volume dimmer; run it on the current scan or the whole cohort.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "settings below here is the N4 bias correction. This is usually necessary only for MRI scans where you have this low-frequency bands of intensity differences.", "startSeconds": 1744.88 }, { "text": "So, if one side of the image looks dimmer than the other side, it's probably this type of bias that you can correct with this option.", "startSeconds": 1757.56 }, { "text": "Now, if you click on this, it will only do it for this image, but if you go ahead and click the one on top here, it will do it for all the scans together.", "startSeconds": 1766.92 }, { "text": "One more step I will suggest to do to", "startSeconds": 1775.64 } ], "algorithm": { "text": "settings below here is the N4 bias correction. This is usually necessary only for MRI scans where you have this low-frequency bands of intensity differences. So, if one side of the image looks dimmer than the other side, it's probably this type of bias that you can correct with this option. Now, if you click on this, it will only do it for this image, but if you go ahead and click the one on top here, it will do it for all the scans together. One more step I will suggest to do to", "math": "", "diagram": "" }, "related": [ "tutorial-ch15", "tutorial-ch17" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1743, "chapterStart": "29:03" }, "images": [] }, { "id": "tutorial-ch17", "title": "Set intensity normalization manually (percentiles and zoom-in tool)", "keywords": [ "tutorial", "transcript", "video", "youtube", "set intensity normalization manually (percentiles and zoom-in tool)" ], "summary": "Normalize the reference with percentile stretching (preferred over histogram matching or peak alignment for most cases), choosing high/low percentiles from the zoomed histogram preview.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "the reference here will be the intensity normalization. As you can see, most of the intensity values right now are on the lower end of your available range of data. So, if we move the slider down, you see that the range gets better and better until we", "startSeconds": 1777.72 }, { "text": "start to see some clipping into black. Okay. So, what I do is I move the slider until I don't see significant clipping, which will be somewhere around there. The only clipping I see now is little pieces of the tongue.", "startSeconds": 1795.32 }, { "text": "By the way, if you want to zoom in, hold the control key and then click and slide.", "startSeconds": 1812.52 }, { "text": "You'll see a zoomed version. Okay, we see this area has some clipping. It's not significant unless you're going to study the tongue, then you would be clipping values of the tongue, then you would probably want to be safer, go up here, let's say right there. So, I think I'm going to do this one. So, what we", "startSeconds": 1818.52 }, { "text": "see now is 99.96% be the top end, and then if we go up here to the intensity normalization options, first I want to only do the scan, so I toggle this that says normalize only current scan. There's many options to do", "startSeconds": 1836.64 }, { "text": "this. Um one is histogram matching. I do not recommend to use this unless your images are very equal to each other, same size, same orientation almost, and same distribution of intensity values. The other one that you could do is align intensity peaks. It finds what are the peaks of the", "startSeconds": 1851.52 }, { "text": "distribution and then tries to shift them until they match. Again, same observation from the one before. I would only do it if they more or less align.", "startSeconds": 1872.92 }, { "text": "Then we have tissue aware normalization. This one tries to find what the peak is and then assume that everything above that peak is going to be bone and then everything below will be soft tissue and", "startSeconds": 1882.12 }, { "text": "then it will try to do both separately. The one in yellow here, percentile normalization, this is the one I suggest doing most of the times. This is going to do a very decent job. What this is going to do is find the percentiles and then stretch all the intensity values to", "startSeconds": 1894.56 }, { "text": "fit the full range. So we go ahead and click it and we have the low percentile as 2%. That's fine since all the values from the background are lumped at the beginning, 2% is almost zero. Then for the high end we should do the same one we found here,", "startSeconds": 1910.88 }, { "text": "99.96. We can just click here and manually set it. Perfect. Now we click apply normalization and we wait. Okay, so what", "startSeconds": 1929.96 } ], "algorithm": { "text": "the reference here will be the intensity normalization. As you can see, most of the intensity values right now are on the lower end of your available range of data. So, if we move the slider down, you see that the range gets better and better until we start to see some clipping into black. Okay. So, what I do is I move the slider until I don't see significant clipping, which will be somewhere around there. The only clipping I see now is little pieces of the tongue. By the way, if you want to zoom in, hold the control key and then click and slide. You'll see a zoomed version. Okay, we see this area has some clipping. It's not significant unless you're going to study the tongue, then you would be clipping values of the tongue, then you would probably want to be safer, go up here, let's say right there. So, I think I'm going to do this one. So, what we see now is 99.96% be the top end, and then if we go up here to the intensity normalization options, first I want to only do the scan, so I toggle this that says normalize only current scan. There's many options to do this. Um one is histogram matching. I do not recommend to use this unless your images are very equal to each other, same size, same orientation almost, and same distribution of intensity values. The other one that you could do is align intensity peaks. It finds what are the peaks of the distribution and then tries to shift them until they match. Again, same observation from the one before. I would only do it if they more or less align. Then we have tissue aware normalization. This one tries to find what the peak is and then assume that everything above that peak is going to be bone and then everything below will be soft tissue and then it will try to do both separately. The one in yellow here, percentile normalization, this is the one I suggest doing most of the times. This is going to do a very decent job. What this is going to do is find the percentiles and then stretch all the intensity values to fit the full range. So we go ahead and click it and we have the low percentile as 2%. That's fine since all the values from the background are lumped at the beginning, 2% is almost zero. Then for the high end we should do the same one we found here, 99.96. We can just click here and manually set it. Perfect. Now we click apply normalization and we wait. Okay, so what", "math": "", "diagram": "" }, "related": [ "tutorial-ch16" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1776, "chapterStart": "29:36" }, "images": [] } ] }, { "id": "tutorial-semi-automated-batch-processing-of-the-rest-of-the-scans", "title": "Semi-automated batch processing of the rest of the scans", "methods": [ { "id": "tutorial-ch18", "title": "Rigid alignment from scan to reference (resampling - rotation - cropping)", "keywords": [ "tutorial", "transcript", "video", "youtube", "rigid alignment from scan to reference (resampling - rotation - cropping)" ], "summary": "With a refined reference and raw subjects, the core value of Aurora is semi-automatically matching subjects to that reference—starting with rigid alignment.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "do we have so far? So far we have a very highly refined reference scan and then two other scans that still have no edits done to them. So if we look at it on the grid view, you can see the reference scan is very nicely processed, oriented and everything, but the other two scans", "startSeconds": 1939.44 }, { "text": "still in their original state. So, here is the biggest value proposition for this software, which is now that you have this reference, you should be able to semi-automatically make the other scans match what the", "startSeconds": 1956.56 }, { "text": "reference looks like. So, the first thing that we need to do for that is alignment.", "startSeconds": 1970.12 } ], "algorithm": { "text": "do we have so far? So far we have a very highly refined reference scan and then two other scans that still have no edits done to them. So if we look at it on the grid view, you can see the reference scan is very nicely processed, oriented and everything, but the other two scans still in their original state. So, here is the biggest value proposition for this software, which is now that you have this reference, you should be able to semi-automatically make the other scans match what the reference looks like. So, the first thing that we need to do for that is alignment.", "math": "", "diagram": "" }, "related": [ "tutorial-ch19" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1939, "chapterStart": "32:19" }, "images": [] }, { "id": "tutorial-ch19", "title": "ALPACA alignment (failed attempt and debugging)", "keywords": [ "tutorial", "transcript", "video", "youtube", "alpaca alignment (failed attempt and debugging)" ], "summary": "Try ALPACA rigid align first (mesh-based, with automatic resampling); when it fails on hard cases, inspect extracted/.../debug alignment figures (PCA, RANSAC, ICP) to see where it broke.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "So, if we go up here to the rigid align to reference, we will see three options.", "startSeconds": 1978.2 }, { "text": "The first one's called Alpaca. It is the one that we mostly recommend to try initially because it's automated and it relies only on the meshes of the images. Therefore, you should be making sure that the threshold is correct so that the meshes of all the scans are good. Once you make sure they're nice,", "startSeconds": 1983.28 }, { "text": "you can go ahead and click Alpaca. One thing to note is that if the voxel sizes, what I name here the spatial resolution, is not the exact same for all the scans. Uh here I just show three decimal places, but in reality, it is very likely that your scans come with many more. What the rigid align to", "startSeconds": 2003.92 }, { "text": "reference step will do is it will actually try to match them by doing a resampling of the voxel space. If we go to the terminal here, we can actually trace this step. So, if we see here, it says voxel size of subject is 0.116, but the", "startSeconds": 2026.08 }, { "text": "reference is 0.118. That means we need to do a resampling of the scan data by a factor of 0.98, etc.", "startSeconds": 2046.04 }, { "text": "So, this is done automatically. You don't need to worry about it. Um it's just nice for me to explain that this is happening. This becomes very important for the later steps that do the elastic registration and all these different things that we want to do in a homogeneous space. Okay, now that it's", "startSeconds": 2055.88 }, { "text": "done, we can go in and check the results. If we click on the scan, here now we can see that it didn't do a good job and that is expected since the embryos are more difficult to register using this Alpaca method. One way that you can check what happened behind the scenes is you can go to the folder where", "startSeconds": 2073.92 }, { "text": "your scans are. You're going to notice this folder called extracted. This folder got generated the moment you ran the extract all scans.", "startSeconds": 2094.56 }, { "text": "Inside this folder, there's everything that happens on the software and it is nicely kept uh separate from your raw scans so that nothing modifies them. Inside here, we can go to one of the scans that we just tried to align and we're going to see a folder called debug alignment. If we go", "startSeconds": 2104.2 }, { "text": "in here, we see some uh figures that got generated for every step of the alignment process.", "startSeconds": 2122.44 }, { "text": "We see the initial cloud of points from the two scans when they were separate from each other. Then the first step is they are centered to their centroids and normalized.", "startSeconds": 2130.6 }, { "text": "Then we see that PCA runs where the main components of variation try to align between the two. Then we do a RANSAC. Uh the RANSAC takes a randomized subset of the points and tries to minimize the distance between the two. Then once that's done, we do three levels of ICP,", "startSeconds": 2141.4 }, { "text": "iterative closest point. First with a low resolution, then a medium resolution, then a high resolution.", "startSeconds": 2161.88 }, { "text": "We see here that it failed all the way every step from the PCA to the RANSAC to the ICP. When we load the grid view, we can see that it failed in both cases.", "startSeconds": 2168.56 }, { "text": "Reason it fails is the RANSAC distributes all the points on the meshes evenly. However, the reality is that for this to succeed, they would need to weigh in the areas that have more idiosyncratic features, more defined anatomy that we can actually match between the two, and weigh less areas", "startSeconds": 2179.24 }, { "text": "that look smooth. Cuz in this case, for example, you have more points or landmarks placed in the smooth areas of the neck than, let's say, here close to the face that is more defined. So, what", "startSeconds": 2200.48 } ], "algorithm": { "text": "So, if we go up here to the rigid align to reference, we will see three options. The first one's called Alpaca. It is the one that we mostly recommend to try initially because it's automated and it relies only on the meshes of the images. Therefore, you should be making sure that the threshold is correct so that the meshes of all the scans are good. Once you make sure they're nice, you can go ahead and click Alpaca. One thing to note is that if the voxel sizes, what I name here the spatial resolution, is not the exact same for all the scans. Uh here I just show three decimal places, but in reality, it is very likely that your scans come with many more. What the rigid align to reference step will do is it will actually try to match them by doing a resampling of the voxel space. If we go to the terminal here, we can actually trace this step. So, if we see here, it says voxel size of subject is 0.116, but the reference is 0.118. That means we need to do a resampling of the scan data by a factor of 0.98, etc. So, this is done automatically. You don't need to worry about it. Um it's just nice for me to explain that this is happening. This becomes very important for the later steps that do the elastic registration and all these different things that we want to do in a homogeneous space. Okay, now that it's done, we can go in and check the results. If we click on the scan, here now we can see that it didn't do a good job and that is expected since the embryos are more difficult to register using this Alpaca method. One way that you can check what happened behind the scenes is you can go to the folder where your scans are. You're going to notice this folder called extracted. This folder got generated the moment you ran the extract all scans. Inside this folder, there's everything that happens on the software and it is nicely kept uh separate from your raw scans so that nothing modifies them. Inside here, we can go to one of the scans that we just tried to align and we're going to see a folder called debug alignment. If we go in here, we see some uh figures that got generated for every step of the alignment process. We see the initial cloud of points from the two scans when they were separate from each other. Then the first step is they are centered to their centroids and normalized. Then we see that PCA runs where the main components of variation try to align between the two. Then we do a RANSAC. Uh the RANSAC takes a randomized subset of the points and tries to minimize the distance between the two. Then once that's done, we do three levels of ICP, iterative closest point. First with a low resolution, then a medium resolution, then a high resolution. We see here that it failed all the way every step from the PCA to the RANSAC to the ICP. When we load the grid view, we can see that it failed in both cases. Reason it fails is the RANSAC distributes all the points on the meshes evenly. However, the reality is that for this to succeed, they would need to weigh in the areas that have more idiosyncratic features, more defined anatomy that we can actually match between the two, and weigh less areas that look smooth. Cuz in this case, for example, you have more points or landmarks placed in the smooth areas of the neck than, let's say, here close to the face that is more defined. So, what", "math": "", "diagram": "" }, "related": [ "tutorial-ch18", "tutorial-ch20" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 1978, "chapterStart": "32:58" }, "images": [] }, { "id": "tutorial-ch20", "title": "Delete edits", "keywords": [ "tutorial", "transcript", "video", "youtube", "delete edits" ], "summary": "Remove failed alignment edits from grid view or the selection trash UI so you can retry another rigid method without leftover transforms.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "you can do now is you can go ahead and delete those edits. Uh there's multiple ways to do this. The first one would be here in grid view, you just click on the trash icon on the right here. Once you click on that, all the edits will appear below each of the scans. Then you can click click, you see", "startSeconds": 2213.08 }, { "text": "how it goes red. That means those edits got selected. Um if you close the grid view, you'll see that there's an analogous a system here where you can check this box here, and then it automatically enables you to select or unselect things that you want", "startSeconds": 2231.08 }, { "text": "to delete. You can select all, select none. Once you're ready to delete them, you just click on the trash bin over here and click okay. Okay, so now we're going to", "startSeconds": 2247.76 } ], "algorithm": { "text": "you can do now is you can go ahead and delete those edits. Uh there's multiple ways to do this. The first one would be here in grid view, you just click on the trash icon on the right here. Once you click on that, all the edits will appear below each of the scans. Then you can click click, you see how it goes red. That means those edits got selected. Um if you close the grid view, you'll see that there's an analogous a system here where you can check this box here, and then it automatically enables you to select or unselect things that you want to delete. You can select all, select none. Once you're ready to delete them, you just click on the trash bin over here and click okay. Okay, so now we're going to", "math": "", "diagram": "" }, "related": [ "tutorial-ch19", "tutorial-ch21" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2212, "chapterStart": "36:52" }, "images": [] }, { "id": "tutorial-ch21", "title": "ANTs rigid alignment (failed attempt)", "keywords": [ "tutorial", "transcript", "video", "youtube", "ants rigid alignment (failed attempt)" ], "summary": "ANTs rigid alignment registers intensity volumes instead of meshes; it can also fail when subjects start far from the reference orientation.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "try the second approach. We're going to try ANTS. ANTS will try to do the registration on the intensity volume images themselves.", "startSeconds": 2257.04 }, { "text": "Okay, now it's finished, and we will check, and we see that it also failed for this method, and that is expected since the images themselves were very out of position between each other, and this makes it harder for them to align with just ANTS. So, we can go ahead and delete them, and now we can go and use", "startSeconds": 2267.36 } ], "algorithm": { "text": "try the second approach. We're going to try ANTS. ANTS will try to do the registration on the intensity volume images themselves. Okay, now it's finished, and we will check, and we see that it also failed for this method, and that is expected since the images themselves were very out of position between each other, and this makes it harder for them to align with just ANTS. So, we can go ahead and delete them, and now we can go and use", "math": "", "diagram": "" }, "related": [ "tutorial-ch20", "tutorial-ch22" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2256, "chapterStart": "37:36" }, "images": [] }, { "id": "tutorial-ch22", "title": "Manual guidepoint-based alignment", "keywords": [ "tutorial", "transcript", "video", "youtube", "manual guidepoint-based alignment" ], "summary": "When automated rigid methods fail, place homologous guide points (start on the reference) and run guidepoint-based alignment for the most accurate rigid match.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "the last method, which is manual guide points. To use manual guide points, we need uh first to set up the guide points on each of the images. This is a process that takes some time, but it's worth it because it will make it the most accurate. For most cases, Alpaca will do", "startSeconds": 2287.56 }, { "text": "a good job. This is one of the cases where it doesn't. So, here I'm going to show you how to set up the guide points. Let's start first with the reference scan. That is very important that you start with this", "startSeconds": 2304.0 }, { "text": "one, and I'll show you why in a second. So, first we're going to go on the right side here, and we're going to find this icon that says place guide points or landmarks.", "startSeconds": 2316.76 }, { "text": "We go in, and we see these two options. We go and click on guide points. Once this mode is active, we can go into full screen to get a better view of this. And now we're simply just going to find places that will have homologous locations in each of the scans. For example, the tip of the nose. That is a", "startSeconds": 2326.96 }, { "text": "very identifiable location in every single scan. And now we're going to tap G on the keyboard, and that will place the guide point right where the mouse is. And you can see on the left here that it got added to a list.", "startSeconds": 2346.2 }, { "text": "In this list, we can delete them, reorder them, etc. Now, let's go ahead and put more guide points. It is important that you put at least three, but the more the better. What you want is to select enough locations that all together align the images on the places that you want. For example, if we place", "startSeconds": 2360.04 }, { "text": "all our guide points near the back of the head, then all the images would align to the back, but probably the front would be misaligned. Conversely, in this case, I'm putting all of them mostly in the front, one in the top, which will make a lot of the anatomy in the back to misalign. But we", "startSeconds": 2379.4 }, { "text": "don't care about that because our final goal will be to add landmarks to the whole face, so we don't really care to analyze the back here. We're focused on the shape of this face. Okay, so now we click save. And now you will see that there's an icon here that says this scan", "startSeconds": 2397.28 }, { "text": "has guide points. Now, what you want to do is go ahead and place guide points in the other two. And since you already placed the guide points in the first one, the process will be guided. What I mean by guided is that you might see here that all the previous guide points that we place here appear from the", "startSeconds": 2415.04 }, { "text": "reference and they're grayed out. So, here what we're going to do is we're going to hold the R key on our keyboard and this will show us the reference. So, here's the reference and we see that the first highlighted guide point is the tip", "startSeconds": 2432.64 }, { "text": "of the nose. So, then we let go of the R key and then we can just go and find that tip of the nose here and then type G.", "startSeconds": 2446.48 }, { "text": "Now, if we wait, automatically it will show us the next guide point. The next guide point being the bottom lip. We place it, automatically it will show us what the next one is. That's the right eye.", "startSeconds": 2455.16 }, { "text": "Okay, so now we go to the right eye. I guess it's it's left eye. Okay, now this is the right eye.", "startSeconds": 2468.08 }, { "text": "Perfect. Now, we go here and then click on there. Now, the top of the head. Perfect. We go to the top of the head. Excellent.", "startSeconds": 2477.0 }, { "text": "So, effectively now we have the same guide points in this specimen. Now, we can just click save and then we can move to the next subject. Okay? So, now we click on the next one. Same, we go to guide points and it will load them from the reference. And if we already memorized where all the guide points are", "startSeconds": 2488.52 }, { "text": "supposed to go, one thing that we can do is go up here to settings in the working directory. You see this icon.", "startSeconds": 2506.48 }, { "text": "There's an option here that says show next guide point preview. We can toggle that off and that will now make it so that we can just place the guide points without it showing us which one's the next.", "startSeconds": 2513.64 }, { "text": "Cuz let's say we already memorized that it's this one, this one, This one, this one. This way you can just go faster. Obviously, if you have more guide points and you are still not familiar with the order, then you should leave that toggled on. Perfect. Now, everything has guide points. Now, we can", "startSeconds": 2527.52 }, { "text": "go ahead and click on the manual guide points alignment and just wait. Okay, now it finished processing. So, now we're going to look at the results.", "startSeconds": 2545.08 }, { "text": "Here we can see that it's successfully aligned the three images. At this point,", "startSeconds": 2557.12 } ], "algorithm": { "text": "the last method, which is manual guide points. To use manual guide points, we need uh first to set up the guide points on each of the images. This is a process that takes some time, but it's worth it because it will make it the most accurate. For most cases, Alpaca will do a good job. This is one of the cases where it doesn't. So, here I'm going to show you how to set up the guide points. Let's start first with the reference scan. That is very important that you start with this one, and I'll show you why in a second. So, first we're going to go on the right side here, and we're going to find this icon that says place guide points or landmarks. We go in, and we see these two options. We go and click on guide points. Once this mode is active, we can go into full screen to get a better view of this. And now we're simply just going to find places that will have homologous locations in each of the scans. For example, the tip of the nose. That is a very identifiable location in every single scan. And now we're going to tap G on the keyboard, and that will place the guide point right where the mouse is. And you can see on the left here that it got added to a list. In this list, we can delete them, reorder them, etc. Now, let's go ahead and put more guide points. It is important that you put at least three, but the more the better. What you want is to select enough locations that all together align the images on the places that you want. For example, if we place all our guide points near the back of the head, then all the images would align to the back, but probably the front would be misaligned. Conversely, in this case, I'm putting all of them mostly in the front, one in the top, which will make a lot of the anatomy in the back to misalign. But we don't care about that because our final goal will be to add landmarks to the whole face, so we don't really care to analyze the back here. We're focused on the shape of this face. Okay, so now we click save. And now you will see that there's an icon here that says this scan has guide points. Now, what you want to do is go ahead and place guide points in the other two. And since you already placed the guide points in the first one, the process will be guided. What I mean by guided is that you might see here that all the previous guide points that we place here appear from the reference and they're grayed out. So, here what we're going to do is we're going to hold the R key on our keyboard and this will show us the reference. So, here's the reference and we see that the first highlighted guide point is the tip of the nose. So, then we let go of the R key and then we can just go and find that tip of the nose here and then type G. Now, if we wait, automatically it will show us the next guide point. The next guide point being the bottom lip. We place it, automatically it will show us what the next one is. That's the right eye. Okay, so now we go to the right eye. I guess it's it's left eye. Okay, now this is the right eye. Perfect. Now, we go here and then click on there. Now, the top of the head. Perfect. We go to the top of the head. Excellent. So, effectively now we have the same guide points in this specimen. Now, we can just click save and then we can move to the next subject. Okay? So, now we click on the next one. Same, we go to guide points and it will load them from the reference. And if we already memorized where all the guide points are supposed to go, one thing that we can do is go up here to settings in the working directory. You see this icon. There's an option here that says show next guide point preview. We can toggle that off and that will now make it so that we can just place the guide points without it showing us which one's the next. Cuz let's say we already memorized that it's this one, this one, This one, this one. This way you can just go faster. Obviously, if you have more guide points and you are still not familiar with the order, then you should leave that toggled on. Perfect. Now, everything has guide points. Now, we can go ahead and click on the manual guide points alignment and just wait. Okay, now it finished processing. So, now we're going to look at the results. Here we can see that it's successfully aligned the three images. At this point,", "math": "", "diagram": "" }, "related": [ "tutorial-ch21", "tutorial-ch23" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2286, "chapterStart": "38:06" }, "images": [] }, { "id": "tutorial-ch23", "title": "Background offset correction automatically", "keywords": [ "tutorial", "transcript", "video", "youtube", "background offset correction automatically" ], "summary": "Once subjects match the reference in size, spacing, and orientation, run automatic background offset on the remaining scans so intensity floors line up.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "all images have the same dimensions, the same voxel size, the same orientation. In other words, they are spatially equivalent.", "startSeconds": 2561.96 }, { "text": "Now, the only remaining things will be the intensity. For that, we can just follow the same steps. We saw that the first one is setting the background. So, we can just go ahead and click on set background automatically.", "startSeconds": 2571.16 }, { "text": "This will automatically set the background for these two images, ignoring the reference, since the reference already has its own. Okay, now it finished and we can check the results quickly. And here we go, the background is all the same level. Now, for the next", "startSeconds": 2585.68 } ], "algorithm": { "text": "all images have the same dimensions, the same voxel size, the same orientation. In other words, they are spatially equivalent. Now, the only remaining things will be the intensity. For that, we can just follow the same steps. We saw that the first one is setting the background. So, we can just go ahead and click on set background automatically. This will automatically set the background for these two images, ignoring the reference, since the reference already has its own. Okay, now it finished and we can check the results quickly. And here we go, the background is all the same level. Now, for the next", "math": "", "diagram": "" }, "related": [ "tutorial-ch22", "tutorial-ch24" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2561, "chapterStart": "42:41" }, "images": [] }, { "id": "tutorial-ch24", "title": "Set intensity normalization automatically (same percentile as reference)", "keywords": [ "tutorial", "transcript", "video", "youtube", "set intensity normalization automatically (same percentile as reference)" ], "summary": "Apply the same percentile normalization used on the reference to all subjects; atypical intensity distributions may need separate treatment.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "step, we are going to do the intensity normalization for all of them. So, we go back here and we click percentile normalization again.", "startSeconds": 2600.56 }, { "text": "And then, if you remember the reference scan, we did it with 99.96. So, we click set volume and then we apply the normalization. It's worth mentioning that one of the three scans here seems to have this blurry part in the center. I think that one will likely work different with the percentile", "startSeconds": 2610.84 }, { "text": "normalization, because for that one, the distribution of intensity values will be different. Therefore, uh 2% and 99.96% will be an actual different value with respect to the others.", "startSeconds": 2634.16 }, { "text": "It's not a big big problem. It's just something to keep in mind. If you see this a lot in your scans, maybe it'll be worth it to treat those ones separately.", "startSeconds": 2651.8 }, { "text": "But again, I don't think is a great great issue. Okay, now it's done. Let's check the results.", "startSeconds": 2663.12 }, { "text": "It looks pretty good. Like I said, this specimen has pretty blurry inside of the head, but since we're just going to use the outside for this purpose, um it should work still. Okay, now if we go", "startSeconds": 2670.4 } ], "algorithm": { "text": "step, we are going to do the intensity normalization for all of them. So, we go back here and we click percentile normalization again. And then, if you remember the reference scan, we did it with 99.96. So, we click set volume and then we apply the normalization. It's worth mentioning that one of the three scans here seems to have this blurry part in the center. I think that one will likely work different with the percentile normalization, because for that one, the distribution of intensity values will be different. Therefore, uh 2% and 99.96% will be an actual different value with respect to the others. It's not a big big problem. It's just something to keep in mind. If you see this a lot in your scans, maybe it'll be worth it to treat those ones separately. But again, I don't think is a great great issue. Okay, now it's done. Let's check the results. It looks pretty good. Like I said, this specimen has pretty blurry inside of the head, but since we're just going to use the outside for this purpose, um it should work still. Okay, now if we go", "math": "", "diagram": "" }, "related": [ "tutorial-ch23", "tutorial-ch25" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2599, "chapterStart": "43:19" }, "images": [] }, { "id": "tutorial-ch25", "title": "Verify all scans are within bounds", "keywords": [ "tutorial", "transcript", "video", "youtube", "verify all scans are within bounds" ], "summary": "In crop mode, use Show all scans to overlay the cohort and confirm nothing sits outside the reference bounds before elastic registration.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "into the reference, here's something you can do to see all the different scans on top of each other. You just go in, go into crop mode, and then click here on show all scans.", "startSeconds": 2685.56 }, { "text": "And then just wait until all of them load. Here now they all loaded, and you can probably not see the the smaller ones that are just inside.", "startSeconds": 2696.4 }, { "text": "But we can see that none of them are falling too far away from this center area. If you wanted to crop further, you can do that here, and then just apply to all scans. But that won't be necessary for this case. So, what is next? Before", "startSeconds": 2706.04 } ], "algorithm": { "text": "into the reference, here's something you can do to see all the different scans on top of each other. You just go in, go into crop mode, and then click here on show all scans. And then just wait until all of them load. Here now they all loaded, and you can probably not see the the smaller ones that are just inside. But we can see that none of them are falling too far away from this center area. If you wanted to crop further, you can do that here, and then just apply to all scans. But that won't be necessary for this case. So, what is next? Before", "math": "", "diagram": "" }, "related": [ "tutorial-ch24", "tutorial-ch26" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2684, "chapterStart": "44:44" }, "images": [] }, { "id": "tutorial-ch26", "title": "Batch cleanup mesh (remove disconnected components)", "keywords": [ "tutorial", "transcript", "video", "youtube", "batch cleanup mesh (remove disconnected components)" ], "summary": "Before elastic registration, batch-cleanup meshes to drop disconnected noise (optional intermediate Gaussian for speckles without permanently blurring saved images).", "options": [], "outputs": [], "transcriptSegments": [ { "text": "we do the most important step, which is the elastic registration, one thing I like to do right before is to run a batch cleanup mesh. The batch cleanup mesh will remove anything that is disconnected from the largest component,", "startSeconds": 2720.8 }, { "text": "in this case the head. So, if you had a noisy patches, things that are leftover anatomy that is just floating away, but it's separate from the main region of interest, then this will collect them and dispose them.", "startSeconds": 2735.0 }, { "text": "Furthermore, if you apply a Gaussian blur, here for example, I can set up 0.8, then what this is going to do is it's just going to apply the Gaussian blur as an intermediate step, and then it's going to check what didn't make it through this filtering. And then, it's going to take that as something to", "startSeconds": 2749.64 }, { "text": "exclude additionally, unless it is connected to the largest component. So, if you have a very noisy image, lots of speckles floating around, you will want to use Gaussian blur before the the batch cleanup mesh. One thing to note is that the Gaussian blur won't be applied", "startSeconds": 2769.28 }, { "text": "to your image. That is to say, the image that is going to be saved as the next step won't be a blurred. It's just used as one of the intermediate steps. With that being said, these scans don't seem to have anything disconnected that's too serious that would affect our analysis. But, uh", "startSeconds": 2785.48 }, { "text": "in any case, I'm just going to run it with a Gaussian blur of zero. Okay, now it's finished, and we can see that all of the scans got the mesh cleaned icon.", "startSeconds": 2805.16 }, { "text": "That means that the algorithm found disconnected components. Otherwise, if it doesn't find anything, it won't create this new edit. You can quickly look at them. They look same as before.", "startSeconds": 2817.04 }, { "text": "As you can see, this one's uh quite smaller than the reference. And then, this one, as we already discussed, has less details internally.", "startSeconds": 2830.16 } ], "algorithm": { "text": "we do the most important step, which is the elastic registration, one thing I like to do right before is to run a batch cleanup mesh. The batch cleanup mesh will remove anything that is disconnected from the largest component, in this case the head. So, if you had a noisy patches, things that are leftover anatomy that is just floating away, but it's separate from the main region of interest, then this will collect them and dispose them. Furthermore, if you apply a Gaussian blur, here for example, I can set up 0.8, then what this is going to do is it's just going to apply the Gaussian blur as an intermediate step, and then it's going to check what didn't make it through this filtering. And then, it's going to take that as something to exclude additionally, unless it is connected to the largest component. So, if you have a very noisy image, lots of speckles floating around, you will want to use Gaussian blur before the the batch cleanup mesh. One thing to note is that the Gaussian blur won't be applied to your image. That is to say, the image that is going to be saved as the next step won't be a blurred. It's just used as one of the intermediate steps. With that being said, these scans don't seem to have anything disconnected that's too serious that would affect our analysis. But, uh in any case, I'm just going to run it with a Gaussian blur of zero. Okay, now it's finished, and we can see that all of the scans got the mesh cleaned icon. That means that the algorithm found disconnected components. Otherwise, if it doesn't find anything, it won't create this new edit. You can quickly look at them. They look same as before. As you can see, this one's uh quite smaller than the reference. And then, this one, as we already discussed, has less details internally.", "math": "", "diagram": "" }, "related": [ "tutorial-ch25", "tutorial-ch27" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2718, "chapterStart": "45:18" }, "images": [] }, { "id": "tutorial-ch27", "title": "Elastic registration options overview", "keywords": [ "tutorial", "transcript", "video", "youtube", "elastic registration options overview" ], "summary": "Configure elastic registration presets (intermediate → aggressive) with affine init, double pass, masking, and optional dice threshold adjustment—hover tooltips explain each control.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "So, now the next step is going to be to elastically align everything to this reference. We can expect that these two cases, which are hard cases, will struggle at some point, but we will see how the results do. Okay, so for the elastic registration, we have this", "startSeconds": 2840.68 }, { "text": "window here. We have many options. If you hover over the options, it tells you what parameters it is using for that registration. I highlight in yellow the one I suggest to do for most of the scenarios.", "startSeconds": 2862.84 }, { "text": "If you run the intermediate and you find that it's still not very good, you can then try aggressive fast. And if that's not enough, you run the aggressive. Why should you go in order?", "startSeconds": 2878.76 }, { "text": "Um because they take a lot of time. So, if time is not an issue for you and you have a powerful computer and you you can leave this running, you can start from a very aggressive registration. So, we have some options here below. The", "startSeconds": 2892.2 }, { "text": "first one is affine initialization. As you can see, this is just using an affine transform that initially aligns everything together. You can scale, shear, and displace the image. It is very recommended that you run this always. Then we have the double pass option. The double pass essentially runs", "startSeconds": 2905.96 }, { "text": "the same registration algorithm again after the first one is done. This is especially useful if certain parts of the image need to match others that are very far away in the other image.", "startSeconds": 2929.32 }, { "text": "So, for example, the scan that we saw that is very small, that with the affine initialization, you would expect it to scale up and almost already match it initially. Then we have use mask. I always recommend using this one. This one uses the threshold you already set", "startSeconds": 2943.96 }, { "text": "up for all the scans to create a mask. And then it will try to register only the elements within the mask and ignore anything in the background. Finally, we have dice adjustment. Dice adjustment will calculate the dice score coefficient after the registration is done, and then it will change the", "startSeconds": 2959.4 }, { "text": "threshold until you get the maximum dice overlap. That is to say, this will change the threshold you already established until you find a better overlap between the two meshes. You can do this one if you realize that the intensity distributions are very different between each other and you", "startSeconds": 2980.4 }, { "text": "want to almost make sure the same amount of content is thresholded. This could be risky though because if the registration is not perfect, then it will overdo this thresholding. So, I recommend leaving it off unless you have a specific case to", "startSeconds": 3000.64 }, { "text": "use this for.", "startSeconds": 3018.72 } ], "algorithm": { "text": "So, now the next step is going to be to elastically align everything to this reference. We can expect that these two cases, which are hard cases, will struggle at some point, but we will see how the results do. Okay, so for the elastic registration, we have this window here. We have many options. If you hover over the options, it tells you what parameters it is using for that registration. I highlight in yellow the one I suggest to do for most of the scenarios. If you run the intermediate and you find that it's still not very good, you can then try aggressive fast. And if that's not enough, you run the aggressive. Why should you go in order? Um because they take a lot of time. So, if time is not an issue for you and you have a powerful computer and you you can leave this running, you can start from a very aggressive registration. So, we have some options here below. The first one is affine initialization. As you can see, this is just using an affine transform that initially aligns everything together. You can scale, shear, and displace the image. It is very recommended that you run this always. Then we have the double pass option. The double pass essentially runs the same registration algorithm again after the first one is done. This is especially useful if certain parts of the image need to match others that are very far away in the other image. So, for example, the scan that we saw that is very small, that with the affine initialization, you would expect it to scale up and almost already match it initially. Then we have use mask. I always recommend using this one. This one uses the threshold you already set up for all the scans to create a mask. And then it will try to register only the elements within the mask and ignore anything in the background. Finally, we have dice adjustment. Dice adjustment will calculate the dice score coefficient after the registration is done, and then it will change the threshold until you get the maximum dice overlap. That is to say, this will change the threshold you already established until you find a better overlap between the two meshes. You can do this one if you realize that the intensity distributions are very different between each other and you want to almost make sure the same amount of content is thresholded. This could be risky though because if the registration is not perfect, then it will overdo this thresholding. So, I recommend leaving it off unless you have a specific case to use this for.", "math": "", "diagram": "" }, "related": [ "tutorial-ch26", "tutorial-ch28" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 2840, "chapterStart": "47:20" }, "images": [] }, { "id": "tutorial-ch28", "title": "A word of caution on double pass", "keywords": [ "tutorial", "transcript", "video", "youtube", "a word of caution on double pass" ], "summary": "Double pass can refine hard-to-reach anatomy but risks overfitting into unrealistic “liquefied” deformations—use it judiciously.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Then we have the double pass. The double pass allows us to run the same registration again on the output of the first one. What this can help us with is refining details that sometimes are too", "startSeconds": 3020.08 }, { "text": "far away or not very easy to align. However, this runs the risk of us over-fitting the two images, uh which results in uh liquefied quality. It's almost as if one image is really forcing itself to try to match the other, which is not realistic. Okay, so the elastic", "startSeconds": 3033.2 } ], "algorithm": { "text": "Then we have the double pass. The double pass allows us to run the same registration again on the output of the first one. What this can help us with is refining details that sometimes are too far away or not very easy to align. However, this runs the risk of us over-fitting the two images, uh which results in uh liquefied quality. It's almost as if one image is really forcing itself to try to match the other, which is not realistic. Okay, so the elastic", "math": "", "diagram": "" }, "related": [ "tutorial-ch27", "tutorial-ch29" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3020, "chapterStart": "50:20" }, "images": [] }, { "id": "tutorial-ch29", "title": "Reviewing elastic registration results (understanding metrics | bad outcomes)", "keywords": [ "tutorial", "transcript", "video", "youtube", "reviewing elastic registration results (understanding metrics | bad outcomes)" ], "summary": "Interpret weak elastic results via surface-proximity % and Dice, per-subject error heatmaps, and reference-averaged registration-error tools (including landmark-colored error).", "options": [], "outputs": [], "transcriptSegments": [ { "text": "registration finished and I decided to do two things. I ran it with poor settings that gave me these bad outcomes and then I'm going to show you the results with better settings. I essentially chose a very fast registration with no double pass,", "startSeconds": 3053.6 }, { "text": "no affine. And as you can see, the performance was poor. So, here we get a percentage.", "startSeconds": 3074.48 }, { "text": "If we hover over the alert here, you can see that for this specific subject, there was 67% alignment of the surface. This is measured by how many vertices from the whole mesh of the outer surface was less", "startSeconds": 3082.56 }, { "text": "than six voxels away from the reference. In this one, we see it's 67%. I set up the threshold of 90% to show you this icon of an alert. Other than that, we also calculate the dice score, which essentially shows you how much of the total area overlaps between the two after registration. But this includes", "startSeconds": 3096.76 }, { "text": "everything internal to the mesh. So, if you're interested in every single part of the image aligning, this is a good one, the dice score. But if you're interested in only the outer surface, then the first one is better. In this", "startSeconds": 3122.16 }, { "text": "one, both of them performed poorly. If you prefer to see the dice score here, you can go to settings and then just switch here to dice and you'll see the result of the dice.", "startSeconds": 3140.4 }, { "text": "We can see that the second one had a nicer dice score, but still not a very good surface distance. Not too bad, though. 86% is not the worst.", "startSeconds": 3150.76 }, { "text": "You have to probably manually check them to verify this. If we click in one of the subjects, we will see the registration error as a heat map on the mesh. So, here you can see that the distance from this mesh to the reference was very poor in many places. Right now, the heat map scale is from 0 mm to 0.04", "startSeconds": 3161.64 }, { "text": "mm, which is the equivalent to 0 to three voxels. If we want to see a wider range, all we need to do is click on the color bar on top here.", "startSeconds": 3184.52 }, { "text": "If we click it once, now we see that we're looking at 0 to about six voxels away.", "startSeconds": 3197.32 }, { "text": "And the millimeter conversion will depend on the voxel sizing of your image. But the number of voxels I show in the bar are always the same. So, here we see it at six, so we click again.", "startSeconds": 3204.64 }, { "text": "0 now to 12, and then we can click again 0 to 24 and then we're back to 0 to 3. Okay, now what if we have hundreds of images and we don't want to go and click on all of them and see how the", "startSeconds": 3219.96 }, { "text": "registration worked on each of them. What we do then is we go to the reference scan. Now we need to go to the place guide points or landmarks mode which is here on the right side of the 3D mesh. Here we see the landmarks option. We click on that and here we", "startSeconds": 3234.28 }, { "text": "have some options on top. The first one says registration error with a button that refreshes and then a switch that if we toggle it on we'll be able to see what is essentially the average of all the registration errors from all the scans. So now that it finished loading we can see the average", "startSeconds": 3252.92 }, { "text": "error. And same as before we can click on the color bar to circle through the different ranges and furthermore what we can do here is we can place our cursor somewhere in the mesh and then tap the key on your keyboard H.", "startSeconds": 3273.0 }, { "text": "When you do that you will automatically load a histogram of what that specific area in the mesh looks like for all the subjects.", "startSeconds": 3292.72 }, { "text": "So since we have only two subjects we only see the two here and we see that both of them specifically where I clicked are 0.5 and 0.4 voxels away.", "startSeconds": 3302.84 }, { "text": "Automatically what happens here is an area of vertices are sampled and then an average distance is calculated. So by default this area is 0.5 voxels but you can change here how big you want that area to be.", "startSeconds": 3315.36 }, { "text": "The other thing you can do is just use a single point. So if instead you just want the single vertex where the cursor over the mesh was, you just change to single, click update, and then you'll see the more", "startSeconds": 3331.08 }, { "text": "specific values. I do not recommend doing single unless you have a very particular scenario just because the meshes can be noisy sometimes. And an individual vertex on the mesh won't represent the actual distance.", "startSeconds": 3344.08 }, { "text": "But if you do an area and it's a small enough area, it will average out some vertices and it will tell you exactly how many. For example, here it's telling me it grabbed 49 vertices and that's what is being averaged. Now that you're here in the histogram, if you click on", "startSeconds": 3360.24 }, { "text": "the bars, it will highlight down here which subject that value belongs to. By default, they are ordered by the largest distance to the smallest distance.", "startSeconds": 3376.96 }, { "text": "So, on top you'll see the worst ones and then on the bottom you'll see the best ones.", "startSeconds": 3387.64 }, { "text": "And the color that they follow will be the exact same color as your current color bar range over here on the top.", "startSeconds": 3392.52 }, { "text": "Now if we go into one of these red areas, we can go ahead and again tap H and here we can see for example that only one of them is very far away, which is the subject. The other one seems fine.", "startSeconds": 3400.44 }, { "text": "So then what you can do is you just click on that subject and it will automatically open it on the preview so you can analyze what went wrong. And here we go and we already saw this one.", "startSeconds": 3414.16 }, { "text": "We can see pretty clearly those areas are of high error. Okay, let's go back to that reference scan and now let's go back to the landmark placement. We go into landmarks and then we load the registration heat map again and since we already had it calculated before, it will load from memory. You can see it", "startSeconds": 3425.52 }, { "text": "loaded pretty fast. If we made some changes and we want to reload it or look at it with a higher detail mesh, then you just go and click this icon here that refreshes the whole calculation. Okay, so now what else can", "startSeconds": 3444.76 }, { "text": "we do here? There is this second switch over here that says landmark registration error.", "startSeconds": 3459.52 }, { "text": "What this is is a tool that colors every landmark that we place on the mesh by this mean registration on a radius around the area. Okay, so now what I'm going to do is I'm going to toggle off the registration heat map. Then I'm going to place some landmarks by using the G key on the keyboard. We can see", "startSeconds": 3466.16 }, { "text": "these landmarks are just appearing in the mesh as red spheres. If we then turn on the registration heat map again, and then we turn on the landmark registration error mode here, now these landmarks get that color of what the registration error is at those areas. If we are confused because they are", "startSeconds": 3485.04 }, { "text": "blending with the heat map in the background, we can just toggle off the registration heat map here, and you can still see them with the [clears throat] color that was assigned. Essentially, this helps us do landmark placement that tries to avoid areas of high error. So, if we turn this back on,", "startSeconds": 3508.88 }, { "text": "we would try to avoid this, for example, and we would try to place landmarks over here. Although, this is just an example, and this level of error is too much.", "startSeconds": 3526.68 }, { "text": "Obviously, this wouldn't be good for you to work with. Okay, now for the more", "startSeconds": 3538.24 } ], "algorithm": { "text": "registration finished and I decided to do two things. I ran it with poor settings that gave me these bad outcomes and then I'm going to show you the results with better settings. I essentially chose a very fast registration with no double pass, no affine. And as you can see, the performance was poor. So, here we get a percentage. If we hover over the alert here, you can see that for this specific subject, there was 67% alignment of the surface. This is measured by how many vertices from the whole mesh of the outer surface was less than six voxels away from the reference. In this one, we see it's 67%. I set up the threshold of 90% to show you this icon of an alert. Other than that, we also calculate the dice score, which essentially shows you how much of the total area overlaps between the two after registration. But this includes everything internal to the mesh. So, if you're interested in every single part of the image aligning, this is a good one, the dice score. But if you're interested in only the outer surface, then the first one is better. In this one, both of them performed poorly. If you prefer to see the dice score here, you can go to settings and then just switch here to dice and you'll see the result of the dice. We can see that the second one had a nicer dice score, but still not a very good surface distance. Not too bad, though. 86% is not the worst. You have to probably manually check them to verify this. If we click in one of the subjects, we will see the registration error as a heat map on the mesh. So, here you can see that the distance from this mesh to the reference was very poor in many places. Right now, the heat map scale is from 0 mm to 0.04 mm, which is the equivalent to 0 to three voxels. If we want to see a wider range, all we need to do is click on the color bar on top here. If we click it once, now we see that we're looking at 0 to about six voxels away. And the millimeter conversion will depend on the voxel sizing of your image. But the number of voxels I show in the bar are always the same. So, here we see it at six, so we click again. 0 now to 12, and then we can click again 0 to 24 and then we're back to 0 to 3. Okay, now what if we have hundreds of images and we don't want to go and click on all of them and see how the registration worked on each of them. What we do then is we go to the reference scan. Now we need to go to the place guide points or landmarks mode which is here on the right side of the 3D mesh. Here we see the landmarks option. We click on that and here we have some options on top. The first one says registration error with a button that refreshes and then a switch that if we toggle it on we'll be able to see what is essentially the average of all the registration errors from all the scans. So now that it finished loading we can see the average error. And same as before we can click on the color bar to circle through the different ranges and furthermore what we can do here is we can place our cursor somewhere in the mesh and then tap the key on your keyboard H. When you do that you will automatically load a histogram of what that specific area in the mesh looks like for all the subjects. So since we have only two subjects we only see the two here and we see that both of them specifically where I clicked are 0.5 and 0.4 voxels away. Automatically what happens here is an area of vertices are sampled and then an average distance is calculated. So by default this area is 0.5 voxels but you can change here how big you want that area to be. The other thing you can do is just use a single point. So if instead you just want the single vertex where the cursor over the mesh was, you just change to single, click update, and then you'll see the more specific values. I do not recommend doing single unless you have a very particular scenario just because the meshes can be noisy sometimes. And an individual vertex on the mesh won't represent the actual distance. But if you do an area and it's a small enough area, it will average out some vertices and it will tell you exactly how many. For example, here it's telling me it grabbed 49 vertices and that's what is being averaged. Now that you're here in the histogram, if you click on the bars, it will highlight down here which subject that value belongs to. By default, they are ordered by the largest distance to the smallest distance. So, on top you'll see the worst ones and then on the bottom you'll see the best ones. And the color that they follow will be the exact same color as your current color bar range over here on the top. Now if we go into one of these red areas, we can go ahead and again tap H and here we can see for example that only one of them is very far away, which is the subject. The other one seems fine. So then what you can do is you just click on that subject and it will automatically open it on the preview so you can analyze what went wrong. And here we go and we already saw this one. We can see pretty clearly those areas are of high error. Okay, let's go back to that reference scan and now let's go back to the landmark placement. We go into landmarks and then we load the registration heat map again and since we already had it calculated before, it will load from memory. You can see it loaded pretty fast. If we made some changes and we want to reload it or look at it with a higher detail mesh, then you just go and click this icon here that refreshes the whole calculation. Okay, so now what else can we do here? There is this second switch over here that says landmark registration error. What this is is a tool that colors every landmark that we place on the mesh by this mean registration on a radius around the area. Okay, so now what I'm going to do is I'm going to toggle off the registration heat map. Then I'm going to place some landmarks by using the G key on the keyboard. We can see these landmarks are just appearing in the mesh as red spheres. If we then turn on the registration heat map again, and then we turn on the landmark registration error mode here, now these landmarks get that color of what the registration error is at those areas. If we are confused because they are blending with the heat map in the background, we can just toggle off the registration heat map here, and you can still see them with the [clears throat] color that was assigned. Essentially, this helps us do landmark placement that tries to avoid areas of high error. So, if we turn this back on, we would try to avoid this, for example, and we would try to place landmarks over here. Although, this is just an example, and this level of error is too much. Obviously, this wouldn't be good for you to work with. Okay, now for the more", "math": "", "diagram": "" }, "related": [ "tutorial-ch28", "tutorial-ch30" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3052, "chapterStart": "50:52" }, "images": [] }, { "id": "tutorial-ch30", "title": "Reviewing elastic registration results (better settings and successful outcomes)", "keywords": [ "tutorial", "transcript", "video", "youtube", "reviewing elastic registration results (better settings and successful outcomes)" ], "summary": "Stronger settings (fine init, double pass, mask) markedly improve proximity and Dice; inspect heatmaps and high-detail meshes to confirm residual error is mostly mesh noise.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "aggressive elastic registration using a fine initialization and double pass and mask, we see a very drastic difference.", "startSeconds": 3542.32 }, { "text": "Now we have a 95% proximity rate to the surface of the reference, a dice coefficient of 83% for the first subject. Then for the second subject, we have 91% proximity and 90% dice score.", "startSeconds": 3552.2 }, { "text": "So, that's pretty decent. And now, let's go ahead and look at it qualitatively. All right, here we have the first subject and it looks pretty decent. As we can see, most of the values lie around less than one voxel away, which is pretty good. Now, if we want to see more detail, we can go ahead and load", "startSeconds": 3567.56 }, { "text": "the high-detail mesh. Now, we can see that the heat map got recalculated with much more details and the areas that seem to have higher error, we can see that they are actually mostly due to the noisy nature of the mesh. Now, we can go", "startSeconds": 3587.88 } ], "algorithm": { "text": "aggressive elastic registration using a fine initialization and double pass and mask, we see a very drastic difference. Now we have a 95% proximity rate to the surface of the reference, a dice coefficient of 83% for the first subject. Then for the second subject, we have 91% proximity and 90% dice score. So, that's pretty decent. And now, let's go ahead and look at it qualitatively. All right, here we have the first subject and it looks pretty decent. As we can see, most of the values lie around less than one voxel away, which is pretty good. Now, if we want to see more detail, we can go ahead and load the high-detail mesh. Now, we can see that the heat map got recalculated with much more details and the areas that seem to have higher error, we can see that they are actually mostly due to the noisy nature of the mesh. Now, we can go", "math": "", "diagram": "" }, "related": [ "tutorial-ch29" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3541, "chapterStart": "59:01" }, "images": [] } ] }, { "id": "tutorial-annotating-and-propagating-landmarks-to-the-whole-dataset", "title": "Annotating and propagating landmarks to the whole dataset", "methods": [ { "id": "tutorial-ch31", "title": "Options for landmarking", "keywords": [ "tutorial", "transcript", "video", "youtube", "options for landmarking" ], "summary": "Landmark either on the reference and propagate via elastic transforms, or work through an atlas—both paths live under the landmark transfer options.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "ahead and start placing some landmarks. There's two options here. We can either go to the reference and place the landmarks there, which then we can just transfer by propagating them using the learned elastic registrations to the", "startSeconds": 3603.88 }, { "text": "subjects. And that is all under this drop-down menu here that says transfer landmarks.", "startSeconds": 3620.24 }, { "text": "Second option we have is to do it with", "startSeconds": 3627.68 } ], "algorithm": { "text": "ahead and start placing some landmarks. There's two options here. We can either go to the reference and place the landmarks there, which then we can just transfer by propagating them using the learned elastic registrations to the subjects. And that is all under this drop-down menu here that says transfer landmarks. Second option we have is to do it with", "math": "", "diagram": "" }, "related": [ "tutorial-ch32" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3603, "chapterStart": "01:00:03" }, "images": [] }, { "id": "tutorial-ch32", "title": "Create an Atlas", "keywords": [ "tutorial", "transcript", "video", "youtube", "create an atlas" ], "summary": "Create an atlas by averaging deformation fields then intensities, yielding a shape-preserving mean image for landmarking when the reference is less ideal.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "an atlas. What happens here is that when we create atlas, I'm going to click it now, it's going to process each of the transformations and generate the average image that actually represents the shape of the average between all of these. So, it finds the average deformation field,", "startSeconds": 3630.6 }, { "text": "essentially bringing each subject to the average position. And then, once all of the scans are in this average position, then we do the intensity averaging.", "startSeconds": 3650.72 }, { "text": "Great. So, now the atlas finished compiling. Now, we can go and click on it. Here, we can see it's obviously just crafted out of these three scans, so it's not going to be a perfect average,", "startSeconds": 3660.6 }, { "text": "but it's a pretty decent-looking one. And we can see the anatomical features are well preserved thanks to this approach, so it's not too blurry. So, what I'm going to do now is I'm going to", "startSeconds": 3671.44 } ], "algorithm": { "text": "an atlas. What happens here is that when we create atlas, I'm going to click it now, it's going to process each of the transformations and generate the average image that actually represents the shape of the average between all of these. So, it finds the average deformation field, essentially bringing each subject to the average position. And then, once all of the scans are in this average position, then we do the intensity averaging. Great. So, now the atlas finished compiling. Now, we can go and click on it. Here, we can see it's obviously just crafted out of these three scans, so it's not going to be a perfect average, but it's a pretty decent-looking one. And we can see the anatomical features are well preserved thanks to this approach, so it's not too blurry. So, what I'm going to do now is I'm going to", "math": "", "diagram": "" }, "related": [ "tutorial-ch31", "tutorial-ch33" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3628, "chapterStart": "01:00:28" }, "images": [] }, { "id": "tutorial-ch33", "title": "Placing landmarks on the reference scan", "keywords": [ "tutorial", "transcript", "video", "youtube", "placing landmarks on the reference scan" ], "summary": "Prefer placing landmarks on the reference (fewer hops than atlas→reference→subjects) while watching registration-error heatmaps; use G to place points and semi-landmarks along mesh geodesics.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "place landmarks in the reference scan. The reason I'm going to do this is it's one less step to transfer landmarks from the reference to subjects. If your reference is decent enough that you can see the features that you want to place landmarks in, then it's better to just", "startSeconds": 3683.92 }, { "text": "use this one. Otherwise, the Atlas needs to internally first transfer the landmarks to a reference and then from reference to the subjects.", "startSeconds": 3701.2 }, { "text": "So, that extra degree of separation could add some slight errors, although nothing to be very concerned about. The other reason I want to use the reference to put the landmarks on is that as we showed before, you can visualize the", "startSeconds": 3710.76 }, { "text": "registration error as you do it. So, here we go. I'm going back to the landmarking system here. I'm going to recalculate the registration heat map cuz this kept the old one where we had", "startSeconds": 3725.16 }, { "text": "those poorly performing registration. Great. So, now that it finished reloading, we can now start placing the landmarks using the G key on our keyboard. For example, here I'm just going to start placing some areas that I know would be of interest here. For", "startSeconds": 3740.12 }, { "text": "example, tip of the nose, bottom lip. And now I want to show you something interesting here, which is let's say I want to place a series of landmarks between two existing landmarks. So, we can click on this one, drag it to the semi landmarks block here, and then click the next one, drag it to the", "startSeconds": 3755.4 }, { "text": "second point here, and then we can just establish how many we want between them.", "startSeconds": 3779.68 }, { "text": "So, here I want to eight semi landmarks, and I just click apply, and then we wait. What this is doing is it is finding the shortest path that follows the outer surface contour of the mesh that connects those two landmarks. Here they are and you can see they are colored differently. They are in blue as", "startSeconds": 3784.04 }, { "text": "opposed to the main landmarks which are in red and we can keep placing landmarks now.", "startSeconds": 3803.6 }, { "text": "So that's it. This is a rough example of what you can achieve. This is obviously not a very good landmarking set. I just made it up for the purposes of this video. Obviously, your expertise in the anatomical literature will guide your", "startSeconds": 3831.24 }, { "text": "placement of the landmarks here. So what I'm going to do now is I'm going to check the landmark registration error.", "startSeconds": 3847.48 }, { "text": "So we're going to turn on registration heat map again. Then I'm going to turn the landmark registration error. Then I'm going to turn off the heat map so I can see them more clearly. And we can see that only a few of them appear not to be completely blue, but they also don't look severely bad in the current", "startSeconds": 3855.56 }, { "text": "range. So I'm happy with how this looks. So now what I will do is click save here. Now that it's done, we can see this logo here on the right side of the bundle that says 98%. What this means is that 98% of the landmarks we just placed are accurately close to the surface in", "startSeconds": 3874.36 }, { "text": "the high-resolution mesh. This is a very sensitive calculation I make here just to keep track of how good the landmarks are positioned. So what we want to do now is we want to make sure they are really snug against the high resolution mesh. To do this, we go into landmark mode, which is distinct from the", "startSeconds": 3897.0 }, { "text": "landmark placement mode. And here we can see each of the individual landmarks and in a different color how far they are from the surface.", "startSeconds": 3919.8 }, { "text": "By the way, if you are color blind, you can just come up here to the settings and then turn on this color blind accessible landmark palette and this will change from green and red to blue, pink, and yellow. So, what we want to do here now is we're going to snap all this to the mesh and for that we have this", "startSeconds": 3930.12 }, { "text": "option the bottom here that says snap landmarks to mesh. Some other options we have in here are random landmarking, which will just place landmarks randomly trying to distribute them evenly on the surface. However, they won't be anatomically", "startSeconds": 3948.72 }, { "text": "paired to any features. Smart landmarking is still under development. Then you can download the mesh, you can download the landmarks, and you can also upload landmarks if you have existing ones. Now that it finished, we can see that now it says 100% on the landmarks in the bundle, but", "startSeconds": 3965.04 }, { "text": "we see that previously it was 98%. So, it keeps both numbers just as a reference. If we go back here to landmark mode, we can see now all of them look color blue, which is zero. And if we switch the color palette, they're all green in here. And if we are curious to see how far they snap from, we can", "startSeconds": 3983.12 }, { "text": "just turn turn on this option that's called snap distance and we can see again some of them look a little more yellow.", "startSeconds": 4004.52 }, { "text": "Going to go to the other palette just so you see, some of them are more magenta.", "startSeconds": 4011.68 }, { "text": "Perfect. Great. So, now that they have been snapped, we can go ahead and", "startSeconds": 4017.24 } ], "algorithm": { "text": "place landmarks in the reference scan. The reason I'm going to do this is it's one less step to transfer landmarks from the reference to subjects. If your reference is decent enough that you can see the features that you want to place landmarks in, then it's better to just use this one. Otherwise, the Atlas needs to internally first transfer the landmarks to a reference and then from reference to the subjects. So, that extra degree of separation could add some slight errors, although nothing to be very concerned about. The other reason I want to use the reference to put the landmarks on is that as we showed before, you can visualize the registration error as you do it. So, here we go. I'm going back to the landmarking system here. I'm going to recalculate the registration heat map cuz this kept the old one where we had those poorly performing registration. Great. So, now that it finished reloading, we can now start placing the landmarks using the G key on our keyboard. For example, here I'm just going to start placing some areas that I know would be of interest here. For example, tip of the nose, bottom lip. And now I want to show you something interesting here, which is let's say I want to place a series of landmarks between two existing landmarks. So, we can click on this one, drag it to the semi landmarks block here, and then click the next one, drag it to the second point here, and then we can just establish how many we want between them. So, here I want to eight semi landmarks, and I just click apply, and then we wait. What this is doing is it is finding the shortest path that follows the outer surface contour of the mesh that connects those two landmarks. Here they are and you can see they are colored differently. They are in blue as opposed to the main landmarks which are in red and we can keep placing landmarks now. So that's it. This is a rough example of what you can achieve. This is obviously not a very good landmarking set. I just made it up for the purposes of this video. Obviously, your expertise in the anatomical literature will guide your placement of the landmarks here. So what I'm going to do now is I'm going to check the landmark registration error. So we're going to turn on registration heat map again. Then I'm going to turn the landmark registration error. Then I'm going to turn off the heat map so I can see them more clearly. And we can see that only a few of them appear not to be completely blue, but they also don't look severely bad in the current range. So I'm happy with how this looks. So now what I will do is click save here. Now that it's done, we can see this logo here on the right side of the bundle that says 98%. What this means is that 98% of the landmarks we just placed are accurately close to the surface in the high-resolution mesh. This is a very sensitive calculation I make here just to keep track of how good the landmarks are positioned. So what we want to do now is we want to make sure they are really snug against the high resolution mesh. To do this, we go into landmark mode, which is distinct from the landmark placement mode. And here we can see each of the individual landmarks and in a different color how far they are from the surface. By the way, if you are color blind, you can just come up here to the settings and then turn on this color blind accessible landmark palette and this will change from green and red to blue, pink, and yellow. So, what we want to do here now is we're going to snap all this to the mesh and for that we have this option the bottom here that says snap landmarks to mesh. Some other options we have in here are random landmarking, which will just place landmarks randomly trying to distribute them evenly on the surface. However, they won't be anatomically paired to any features. Smart landmarking is still under development. Then you can download the mesh, you can download the landmarks, and you can also upload landmarks if you have existing ones. Now that it finished, we can see that now it says 100% on the landmarks in the bundle, but we see that previously it was 98%. So, it keeps both numbers just as a reference. If we go back here to landmark mode, we can see now all of them look color blue, which is zero. And if we switch the color palette, they're all green in here. And if we are curious to see how far they snap from, we can just turn turn on this option that's called snap distance and we can see again some of them look a little more yellow. Going to go to the other palette just so you see, some of them are more magenta. Perfect. Great. So, now that they have been snapped, we can go ahead and", "math": "", "diagram": "" }, "related": [ "tutorial-ch32", "tutorial-ch34" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 3682, "chapterStart": "01:01:22" }, "images": [] }, { "id": "tutorial-ch34", "title": "Transfer landmarks from reference to subjects", "keywords": [ "tutorial", "transcript", "video", "youtube", "transfer landmarks from reference to subjects" ], "summary": "Transfer reference landmarks to subjects with snap-to-mesh and the learned elastic warps, then review placement quality on each subject.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "transfer them to all our subjects. To do this, we have this option up here. It's called transfer landmarks, and we will toggle on the option parameter called snap landmarks to mesh.", "startSeconds": 4022.24 }, { "text": "This way automatically after they are transferred, they will snap to the surface.", "startSeconds": 4036.2 }, { "text": "And now we're going to run reference to subject. So now what we found here is that the first subject performed pretty decently. So now what we find here with the results is originally the first subject performed at a 72% rate, and it says here when we hover that 15 landmarks were displaced more than one", "startSeconds": 4042.32 }, { "text": "voxel. So this percentage represents the success rate, but after the snapping we see that it's now at 100%.", "startSeconds": 4063.76 }, { "text": "For the second subject, this was 87%. Now, the landmarks were transferred to the edit one before the elastic registration. This means that the marks are placed in this one, which is the third edit called mesh cleaned. So as you can see, we are at the edit number three now, and it has a little icon on", "startSeconds": 4073.76 }, { "text": "the bottom that indicates that this is the one that has the landmarks. Now, if we go into landmark mode, we're able to see all the landmarks, and they appear to all be color blue, which means they are all tight against the mesh. If we click on snap distance, as we saw before, we can see how far each of the", "startSeconds": 4095.72 }, { "text": "landmarks had to snap from. So we see there's a couple that are here on top that snapped from a distance around two voxels, which is not too bad.", "startSeconds": 4114.32 }, { "text": "Now, something you might want to check is imagine that your data set is composed of 100 subjects, and you just transferred landmarks to all of them, and you don't want to click one by one to check if any of the landmarks snapped from way too far away. Here we can go um back to the landmark mode. Here we", "startSeconds": 4123.76 }, { "text": "click on snap distance and then we click on show average from all, which is this option right beside the snap distance.", "startSeconds": 4143.56 }, { "text": "And now you can see that the color changed. So these colors now represent the average distance that each of the landmarks had to snap from. Let's change the range from this to four voxels, okay?", "startSeconds": 4150.84 }, { "text": "That's pretty good. If you are concerned about any of the landmarks, for example, let's say this one over here seems to be one of the highest distances. So we just click on it and it will display a histogram of that specific landmark for all the different subjects. So we can", "startSeconds": 4164.96 }, { "text": "see here that in this subject the snapping distance was 1.64 voxels away, which is pretty good.", "startSeconds": 4183.04 }, { "text": "And then for the other one was just one voxel. This can help you spot outliers, subjects that maybe somewhere along the way something didn't perform as well and it evaded all the checks before.", "startSeconds": 4190.96 }, { "text": "And now at this stage, which is the data that you are actually interested in, which is the landmarks, will have the necessary information to spot this outliers. Finally, what we want to do", "startSeconds": 4205.2 } ], "algorithm": { "text": "transfer them to all our subjects. To do this, we have this option up here. It's called transfer landmarks, and we will toggle on the option parameter called snap landmarks to mesh. This way automatically after they are transferred, they will snap to the surface. And now we're going to run reference to subject. So now what we found here is that the first subject performed pretty decently. So now what we find here with the results is originally the first subject performed at a 72% rate, and it says here when we hover that 15 landmarks were displaced more than one voxel. So this percentage represents the success rate, but after the snapping we see that it's now at 100%. For the second subject, this was 87%. Now, the landmarks were transferred to the edit one before the elastic registration. This means that the marks are placed in this one, which is the third edit called mesh cleaned. So as you can see, we are at the edit number three now, and it has a little icon on the bottom that indicates that this is the one that has the landmarks. Now, if we go into landmark mode, we're able to see all the landmarks, and they appear to all be color blue, which means they are all tight against the mesh. If we click on snap distance, as we saw before, we can see how far each of the landmarks had to snap from. So we see there's a couple that are here on top that snapped from a distance around two voxels, which is not too bad. Now, something you might want to check is imagine that your data set is composed of 100 subjects, and you just transferred landmarks to all of them, and you don't want to click one by one to check if any of the landmarks snapped from way too far away. Here we can go um back to the landmark mode. Here we click on snap distance and then we click on show average from all, which is this option right beside the snap distance. And now you can see that the color changed. So these colors now represent the average distance that each of the landmarks had to snap from. Let's change the range from this to four voxels, okay? That's pretty good. If you are concerned about any of the landmarks, for example, let's say this one over here seems to be one of the highest distances. So we just click on it and it will display a histogram of that specific landmark for all the different subjects. So we can see here that in this subject the snapping distance was 1.64 voxels away, which is pretty good. And then for the other one was just one voxel. This can help you spot outliers, subjects that maybe somewhere along the way something didn't perform as well and it evaded all the checks before. And now at this stage, which is the data that you are actually interested in, which is the landmarks, will have the necessary information to spot this outliers. Finally, what we want to do", "math": "", "diagram": "" }, "related": [ "tutorial-ch33", "tutorial-ch35" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4020, "chapterStart": "01:07:00" }, "images": [] }, { "id": "tutorial-ch35", "title": "Export all landmarks as a .csv file", "keywords": [ "tutorial", "transcript", "video", "youtube", "export all landmarks as a .csv file" ], "summary": "Export all landmarks as a CSV for Excel or downstream morphometrics; files are written into the project’s organized output structure.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "here is export all the landmarks as a CSV file. So this is a table that you can open with Excel or any other program that can read CSVs. So what to you do is you can just click on this option here and it will download a zip folder. When we open this folder, we can see three CSVs got", "startSeconds": 4216.24 }, { "text": "generated. The important one is this one that says landmarks. Other than that, we have two CSVs that end with the word distances.", "startSeconds": 4237.84 }, { "text": "Each of these ones will show you how far each landmark is from the surface mesh.", "startSeconds": 4248.4 }, { "text": "This one shows the current one. If they were snapped, you will expect this one to be close to zero for every single landmark.", "startSeconds": 4253.48 }, { "text": "And for this one, you will see all of them pre-snapping. You can get the raw information of how far they were.", "startSeconds": 4261.32 }, { "text": "So, the same things we were looking at here in the interface, now you can have them as data.", "startSeconds": 4268.72 }, { "text": "This can help you generate statistical analysis that determine which parts of your landmarking schemes have more confidence or less. So, let's go ahead and open this CSV. So, here we are inside of Excel and we can see that we have the three subjects, reference and", "startSeconds": 4274.4 }, { "text": "the other two. And we can see that for each landmark, there is X, Y, and Z. So, that is the format chosen here.", "startSeconds": 4293.44 }, { "text": "You can obviously load this into any program of your choice. Now, we go to", "startSeconds": 4303.24 } ], "algorithm": { "text": "here is export all the landmarks as a CSV file. So this is a table that you can open with Excel or any other program that can read CSVs. So what to you do is you can just click on this option here and it will download a zip folder. When we open this folder, we can see three CSVs got generated. The important one is this one that says landmarks. Other than that, we have two CSVs that end with the word distances. Each of these ones will show you how far each landmark is from the surface mesh. This one shows the current one. If they were snapped, you will expect this one to be close to zero for every single landmark. And for this one, you will see all of them pre-snapping. You can get the raw information of how far they were. So, the same things we were looking at here in the interface, now you can have them as data. This can help you generate statistical analysis that determine which parts of your landmarking schemes have more confidence or less. So, let's go ahead and open this CSV. So, here we are inside of Excel and we can see that we have the three subjects, reference and the other two. And we can see that for each landmark, there is X, Y, and Z. So, that is the format chosen here. You can obviously load this into any program of your choice. Now, we go to", "math": "", "diagram": "" }, "related": [ "tutorial-ch34" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4216, "chapterStart": "01:10:16" }, "images": [] } ] }, { "id": "tutorial-segmentation-module", "title": "Segmentation module", "methods": [ { "id": "tutorial-ch36", "title": "Segmentation tools overview", "keywords": [ "tutorial", "transcript", "video", "youtube", "segmentation tools overview" ], "summary": "Open the segmentation module to paint masks on the reference, choose labels/brushes, and prepare tissue for AI fill, masking, and later landmarking on isolated anatomy.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "the bottom here where we can see the intensity value images in the three views. And we can go to the bottom right and click on the wrench icon.", "startSeconds": 4309.04 }, { "text": "This will show up some of the options available. The first option says segment mode. So, we click on this one and this will load the full suite that allows us to generate segmentations.", "startSeconds": 4321.12 }, { "text": "By default, the image that is loaded it's the same as you see over here, which is a downscaled compressed version of the original image. So, if you wanted to work with the higher resolution image, all you need to do is click on this button over here that says full resolution. Perfect. So, now we can see", "startSeconds": 4335.6 }, { "text": "the higher resolution image. Just a note, this takes a lot of memory and most computers won't be able to handle it. Only if you have a powerful computer, this is a useful way to look at your image. Otherwise, you will", "startSeconds": 4356.24 }, { "text": "face some lagging and slow downs. In which case using the half resolution compressed image is just as okay. You just get less fine grain detail. So here I went back to the half resolution just to make things uh more efficient for the demo. So what you do here is first you can switch to threshold mode", "startSeconds": 4371.48 }, { "text": "and in threshold mode you can move around the threshold to try to separate out the anatomy that you want to segment first. So one thing I want to do here is segment the brain.", "startSeconds": 4395.28 }, { "text": "So I'm going to locate myself in a slice that contains as much brain as I can see for the current sagittal view. Now I want to move the threshold until it pretty much covers the area that I want to segment, but it feels like it's separate enough from surrounding anatomy. Right now to start I think the", "startSeconds": 4405.32 }, { "text": "original one that we had works pretty decently. So now what you can see is the the brush here in red as a circle. We can change its size by using the scroll wheel.", "startSeconds": 4426.68 }, { "text": "So we can scroll up and get a bigger size. We can see in the top left here it says brush size 85 pixels and then what", "startSeconds": 4437.72 } ], "algorithm": { "text": "the bottom here where we can see the intensity value images in the three views. And we can go to the bottom right and click on the wrench icon. This will show up some of the options available. The first option says segment mode. So, we click on this one and this will load the full suite that allows us to generate segmentations. By default, the image that is loaded it's the same as you see over here, which is a downscaled compressed version of the original image. So, if you wanted to work with the higher resolution image, all you need to do is click on this button over here that says full resolution. Perfect. So, now we can see the higher resolution image. Just a note, this takes a lot of memory and most computers won't be able to handle it. Only if you have a powerful computer, this is a useful way to look at your image. Otherwise, you will face some lagging and slow downs. In which case using the half resolution compressed image is just as okay. You just get less fine grain detail. So here I went back to the half resolution just to make things uh more efficient for the demo. So what you do here is first you can switch to threshold mode and in threshold mode you can move around the threshold to try to separate out the anatomy that you want to segment first. So one thing I want to do here is segment the brain. So I'm going to locate myself in a slice that contains as much brain as I can see for the current sagittal view. Now I want to move the threshold until it pretty much covers the area that I want to segment, but it feels like it's separate enough from surrounding anatomy. Right now to start I think the original one that we had works pretty decently. So now what you can see is the the brush here in red as a circle. We can change its size by using the scroll wheel. So we can scroll up and get a bigger size. We can see in the top left here it says brush size 85 pixels and then what", "math": "", "diagram": "" }, "related": [ "tutorial-ch37" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4308, "chapterStart": "01:11:48" }, "images": [] }, { "id": "tutorial-ch37", "title": "Manually segmenting a middle slice in the three axes views", "keywords": [ "tutorial", "transcript", "video", "youtube", "manually segmenting a middle slice in the three axes views" ], "summary": "Seed a 3D mask by painting mid-slices in all three axes with thresholded/regular brushes, eraser, undo, and optional 3D brush—then save before expanding with AI.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "we can do is we can switch back to range mode so we can see the actual intensity values. And by default the option that is on is called thresholded brush.", "startSeconds": 4447.52 }, { "text": "The thresholded brush as the name suggests will be a brush that if you paint through it will only keep the areas that are above the threshold.", "startSeconds": 4460.32 }, { "text": "So we can do the bulk of this slice of the brain by filling up the inside without going into the outer edges yet.", "startSeconds": 4470.36 }, { "text": "I'm going to do the bottom ones because those ones are well defined and nice. We can go down here.", "startSeconds": 4480.28 }, { "text": "I can make the brush smaller. Now I can go into the finer details of the edges. If you have a tablet with a pen, it's also easier to use that. Now we can refine details here. So to refine details, we can just change to a regular brush. In the regular brush, we can just", "startSeconds": 4493.84 }, { "text": "fill these gaps that appeared down here in the stem. And now let's say we screwed up and we painted outside of the boundaries. We can backtrack this mistake by clicking this button over here, undo.", "startSeconds": 4511.32 }, { "text": "And finally here I'm going to use a razor and I'm going to go to a very small size and fix these details out here.", "startSeconds": 4538.2 }, { "text": "Okay, so now I am happy with this slice. Now instead of going through all of the slices, which would take you forever as you can see here, there are above 300.", "startSeconds": 4567.4 }, { "text": "What you can do is simply switch to the axial view. Here you can see it as a line. Now we can go ahead and fill up this other view.", "startSeconds": 4578.0 }, { "text": "Okay, so now we go to the final view. Okay, so now that you're happy with the three views, what you need to do is go and click save mask. Once it is saved, now you can do a myriad of operations with it. Before I go into them, I want to show you some other options that we", "startSeconds": 4600.88 }, { "text": "have here. First one is the 3D brush. So, when we turn it on, what it does is your brush can now paint across the slices. And the number of slices it can paint through will be equivalent to the same radius the brush is.", "startSeconds": 4634.52 }, { "text": "So, for example, I'm going to paint here the tongue, and you will see that in the other views, if we align the crosshair here, you can see that it painted through.", "startSeconds": 4652.04 }, { "text": "If I just paint one circle, the circle becomes a cylinder that paints through all the slices. I'm going to undo this.", "startSeconds": 4660.64 }, { "text": "The 3D brush also works for the other brush options and eraser, for example. Something else you can do is you can apply the threshold completely to the whole image.", "startSeconds": 4668.36 }, { "text": "So, if you do this, you will see that everything becomes masked that is above the threshold.", "startSeconds": 4679.84 }, { "text": "So, this would be useful if you want to, instead of brush in, you want to erase out the mask that you're trying to get to. So, I'm going to undo this operation.", "startSeconds": 4685.96 } ], "algorithm": { "text": "we can do is we can switch back to range mode so we can see the actual intensity values. And by default the option that is on is called thresholded brush. The thresholded brush as the name suggests will be a brush that if you paint through it will only keep the areas that are above the threshold. So we can do the bulk of this slice of the brain by filling up the inside without going into the outer edges yet. I'm going to do the bottom ones because those ones are well defined and nice. We can go down here. I can make the brush smaller. Now I can go into the finer details of the edges. If you have a tablet with a pen, it's also easier to use that. Now we can refine details here. So to refine details, we can just change to a regular brush. In the regular brush, we can just fill these gaps that appeared down here in the stem. And now let's say we screwed up and we painted outside of the boundaries. We can backtrack this mistake by clicking this button over here, undo. And finally here I'm going to use a razor and I'm going to go to a very small size and fix these details out here. Okay, so now I am happy with this slice. Now instead of going through all of the slices, which would take you forever as you can see here, there are above 300. What you can do is simply switch to the axial view. Here you can see it as a line. Now we can go ahead and fill up this other view. Okay, so now we go to the final view. Okay, so now that you're happy with the three views, what you need to do is go and click save mask. Once it is saved, now you can do a myriad of operations with it. Before I go into them, I want to show you some other options that we have here. First one is the 3D brush. So, when we turn it on, what it does is your brush can now paint across the slices. And the number of slices it can paint through will be equivalent to the same radius the brush is. So, for example, I'm going to paint here the tongue, and you will see that in the other views, if we align the crosshair here, you can see that it painted through. If I just paint one circle, the circle becomes a cylinder that paints through all the slices. I'm going to undo this. The 3D brush also works for the other brush options and eraser, for example. Something else you can do is you can apply the threshold completely to the whole image. So, if you do this, you will see that everything becomes masked that is above the threshold. So, this would be useful if you want to, instead of brush in, you want to erase out the mask that you're trying to get to. So, I'm going to undo this operation.", "math": "", "diagram": "" }, "related": [ "tutorial-ch36", "tutorial-ch38" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4446, "chapterStart": "01:14:06" }, "images": [] }, { "id": "tutorial-ch38", "title": "AI-based 3D segmentation (MedSAM2)", "keywords": [ "tutorial", "transcript", "video", "youtube", "ai-based 3d segmentation (medsam2)" ], "summary": "Run MedSAM2 to expand the three mid-slice seeds into a full volume via per-axis propagations and majority voting at each voxel.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "And now, as you can see, we only segmented an intermediate slice for each of the three views.", "startSeconds": 4696.28 }, { "text": "So, now if we want to get the full brain, one thing that we're going to do is we're going to use AI-based segmentation with a model that is called Net Sam 2.", "startSeconds": 4704.2 }, { "text": "So, you can find that here in the right of the window. We choose the model and we just click run. This will use the mask that you saved previously and it will pass it through the AI algorithm.", "startSeconds": 4716.04 }, { "text": "And what it does in a basic sense is go slice by slice and then expand out until it fills the volume.", "startSeconds": 4730.68 }, { "text": "And that volume will be delimited by the extension of each of the three different view masks that already exist. In the end, this ends up becoming into three different segmentations of the whole brain.", "startSeconds": 4739.24 }, { "text": "After which there is a voting system that on each voxel, it asks the question, are two or more of the segmentations including this voxel? If the answer is yes, then that will be included in the final segmentation. Now, we can see that the segmentation was loaded correctly in three dimensions. It", "startSeconds": 4754.28 }, { "text": "performed pretty decently. There There might still be some inaccuracies here and there, but overall, it did a great", "startSeconds": 4775.68 } ], "algorithm": { "text": "And now, as you can see, we only segmented an intermediate slice for each of the three views. So, now if we want to get the full brain, one thing that we're going to do is we're going to use AI-based segmentation with a model that is called Net Sam 2. So, you can find that here in the right of the window. We choose the model and we just click run. This will use the mask that you saved previously and it will pass it through the AI algorithm. And what it does in a basic sense is go slice by slice and then expand out until it fills the volume. And that volume will be delimited by the extension of each of the three different view masks that already exist. In the end, this ends up becoming into three different segmentations of the whole brain. After which there is a voting system that on each voxel, it asks the question, are two or more of the segmentations including this voxel? If the answer is yes, then that will be included in the final segmentation. Now, we can see that the segmentation was loaded correctly in three dimensions. It performed pretty decently. There There might still be some inaccuracies here and there, but overall, it did a great", "math": "", "diagram": "" }, "related": [ "tutorial-ch37", "tutorial-ch39" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4696, "chapterStart": "01:18:16" }, "images": [] }, { "id": "tutorial-ch39", "title": "Visualize the segmentation preview as a mesh", "keywords": [ "tutorial", "transcript", "video", "youtube", "visualize the segmentation preview as a mesh" ], "summary": "Preview the saved mask as a mesh (mask vs threshold display, optional blur), noting you must apply the mask to the image before landmarking on that tissue.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "job. What you can do now is since this automatically saves, you can exit this window and visualize it separately. To do that, we need to refresh the website, go back to the reference. To visualize the created segmentation, we can just go into the refresh mode and within the", "startSeconds": 4783.12 }, { "text": "option mask label, now we will see there is label one. If we select label one and then we wait, now what we see is a preview of what we just segmented using the three slices and then the AI approach.", "startSeconds": 4802.44 }, { "text": "As you can see, this looks pretty decent. We can also see a preview of that in the voxel intensities down here.", "startSeconds": 4819.4 }, { "text": "However, this is not applied to the edit. Now, if we want to smooth the mesh to prevent these jagged lines that we see from the segmentation. One thing we can do is go into the refresh mode.", "startSeconds": 4826.84 }, { "text": "And now we see that currently it is set up to mask. So, what this does is the 3D mesh is created out of the mask and not the actual thresholded intensities.", "startSeconds": 4839.32 }, { "text": "If we switch it to threshold and then refresh, now it it looks slightly smoother because now it's using the actual threshold for the area within the mask. And if we want to go further, we can just go down here and add a Gaussian blur, then click refresh. And now we have a very smooth surface that we can", "startSeconds": 4852.56 }, { "text": "visualize and explore. But, let's say you want to add landmarks to this. So, you have this organ is the brain and then you want to study its shape, so you go to the landmarks mode, but now I show you this message here that says that to add landmarks, you must apply the mask", "startSeconds": 4873.48 }, { "text": "to the image first. How do we do that? Okay. So, we go back to the segmentation", "startSeconds": 4890.6 } ], "algorithm": { "text": "job. What you can do now is since this automatically saves, you can exit this window and visualize it separately. To do that, we need to refresh the website, go back to the reference. To visualize the created segmentation, we can just go into the refresh mode and within the option mask label, now we will see there is label one. If we select label one and then we wait, now what we see is a preview of what we just segmented using the three slices and then the AI approach. As you can see, this looks pretty decent. We can also see a preview of that in the voxel intensities down here. However, this is not applied to the edit. Now, if we want to smooth the mesh to prevent these jagged lines that we see from the segmentation. One thing we can do is go into the refresh mode. And now we see that currently it is set up to mask. So, what this does is the 3D mesh is created out of the mask and not the actual thresholded intensities. If we switch it to threshold and then refresh, now it it looks slightly smoother because now it's using the actual threshold for the area within the mask. And if we want to go further, we can just go down here and add a Gaussian blur, then click refresh. And now we have a very smooth surface that we can visualize and explore. But, let's say you want to add landmarks to this. So, you have this organ is the brain and then you want to study its shape, so you go to the landmarks mode, but now I show you this message here that says that to add landmarks, you must apply the mask to the image first. How do we do that? Okay. So, we go back to the segmentation", "math": "", "diagram": "" }, "related": [ "tutorial-ch38", "tutorial-ch40" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4783, "chapterStart": "01:19:43" }, "images": [] }, { "id": "tutorial-ch40", "title": "Additional manual segmentation options", "keywords": [ "tutorial", "transcript", "video", "youtube", "additional manual segmentation options" ], "summary": "Extra segmentation tools: invert threshold for negative-space cuts, add multiple labels, show all labels together, or delete unused labels.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "mode. Now there's a shortcut to get there since there's a mask present and it's going to be this icon over here that says mask available for current edit.", "startSeconds": 4896.04 }, { "text": "So, we just click on it. Now we're back here and before we do anything, I want to show you two more things about this interface. The first one is that we have an option that it is called invert threshold.", "startSeconds": 4904.76 }, { "text": "So, if you go into threshold mode, you invert the threshold. Now you can create masks on the negative spaces.", "startSeconds": 4917.44 }, { "text": "This can be useful to create specific cuts around objects that you can then clear out using the clean mesh option over here. So, this is a way to do it.", "startSeconds": 4926.28 }, { "text": "Just go and cut up the negative spaces and then mask them out. Okay, the second thing I want to show you is you can add more labels. Right now, we just have this label one, but you can click in the plus icon over here, which adds a second label. So, now if we go back to the", "startSeconds": 4938.56 }, { "text": "first label, it's right there, didn't move. We go to label two, we can now create a mask, let's say of, I don't know, the tongue or some other part of the head. And we can save this again, and now we would have the two labels to work with. Something you can do as well", "startSeconds": 4952.44 }, { "text": "here is show all labels. So, if you want to see everything all together, it will take a different color, one for each object.", "startSeconds": 4968.08 }, { "text": "And you can also fully delete the label. So, I can just delete the label two right here.", "startSeconds": 4975.52 }, { "text": "And I'll click okay. And now we're back to this. Perfect. So, now to the masking. Two", "startSeconds": 4982.8 } ], "algorithm": { "text": "mode. Now there's a shortcut to get there since there's a mask present and it's going to be this icon over here that says mask available for current edit. So, we just click on it. Now we're back here and before we do anything, I want to show you two more things about this interface. The first one is that we have an option that it is called invert threshold. So, if you go into threshold mode, you invert the threshold. Now you can create masks on the negative spaces. This can be useful to create specific cuts around objects that you can then clear out using the clean mesh option over here. So, this is a way to do it. Just go and cut up the negative spaces and then mask them out. Okay, the second thing I want to show you is you can add more labels. Right now, we just have this label one, but you can click in the plus icon over here, which adds a second label. So, now if we go back to the first label, it's right there, didn't move. We go to label two, we can now create a mask, let's say of, I don't know, the tongue or some other part of the head. And we can save this again, and now we would have the two labels to work with. Something you can do as well here is show all labels. So, if you want to see everything all together, it will take a different color, one for each object. And you can also fully delete the label. So, I can just delete the label two right here. And I'll click okay. And now we're back to this. Perfect. So, now to the masking. Two", "math": "", "diagram": "" }, "related": [ "tutorial-ch39", "tutorial-ch41" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4894, "chapterStart": "01:21:34" }, "images": [] }, { "id": "tutorial-ch41", "title": "Mask out or apply mask to image", "keywords": [ "tutorial", "transcript", "video", "youtube", "mask out or apply mask to image" ], "summary": "Mask out deletes voxels inside a label; Apply mask to image isolates tissue (e.g. brain) into a new masks folder scan that keeps prior transforms for landmark propagation.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "options here. The first one is mask out label one. This will effectively create a new edit in which all the voxels within this mask are deleted. They will just be assigned a background value. This is what you would do if you were creating cuts between the organs, so that you can then", "startSeconds": 4989.84 }, { "text": "clear them out using the clean mesh. The other thing you can do is that if you want to isolate the brain, for example, and put landmarks on it, there's this option here that says apply mask to image.", "startSeconds": 5009.08 }, { "text": "This will save a masked scan. So, we can go ahead and click that. And this will prompt you to give a name. So, here I'm just going to type brain, and then click on apply and save. Once done, we can go ahead and close this and refresh the website. And now, what we can see is there's an option that", "startSeconds": 5022.28 }, { "text": "appears up here that is called masked available. Behind the scenes, what happened is if we click this bundle option off, we can see that there are several folders in here. One of them is called masks. [clears throat] If we click on masks, we'll see there's brain. So, I'm just going to go back", "startSeconds": 5042.48 }, { "text": "here. We can just click on this button and shortcut straight there. So, we'll go straight into this path that now says brain. Perfect. So, now we can click on this folder brain and we will see that this subject we clicked to mask out appears here. So, if we click on this we can see the extracted brain", "startSeconds": 5059.96 }, { "text": "appears immediately. And now this is treated just as any other scan. So, now we can go ahead and look into placing landmarks and the benefit will be that this inherits all the elastic transformations and it inherits all the positioning and all the work we did", "startSeconds": 5083.96 }, { "text": "before. So, if we apply the masks to all of the different subjects, we should be able to straight away place landmarks and propagate them. So, let's do that. Let's walk you through that process. So, before I even place landmarks, we need to get the rest of the subjects over", "startSeconds": 5103.84 }, { "text": "here. How do we do that? Well, the same place where the button that appeared before that said available masks was now shows go to main folder.", "startSeconds": 5122.4 }, { "text": "Now, we're back here. So, the first", "startSeconds": 5133.2 } ], "algorithm": { "text": "options here. The first one is mask out label one. This will effectively create a new edit in which all the voxels within this mask are deleted. They will just be assigned a background value. This is what you would do if you were creating cuts between the organs, so that you can then clear them out using the clean mesh. The other thing you can do is that if you want to isolate the brain, for example, and put landmarks on it, there's this option here that says apply mask to image. This will save a masked scan. So, we can go ahead and click that. And this will prompt you to give a name. So, here I'm just going to type brain, and then click on apply and save. Once done, we can go ahead and close this and refresh the website. And now, what we can see is there's an option that appears up here that is called masked available. Behind the scenes, what happened is if we click this bundle option off, we can see that there are several folders in here. One of them is called masks. [clears throat] If we click on masks, we'll see there's brain. So, I'm just going to go back here. We can just click on this button and shortcut straight there. So, we'll go straight into this path that now says brain. Perfect. So, now we can click on this folder brain and we will see that this subject we clicked to mask out appears here. So, if we click on this we can see the extracted brain appears immediately. And now this is treated just as any other scan. So, now we can go ahead and look into placing landmarks and the benefit will be that this inherits all the elastic transformations and it inherits all the positioning and all the work we did before. So, if we apply the masks to all of the different subjects, we should be able to straight away place landmarks and propagate them. So, let's do that. Let's walk you through that process. So, before I even place landmarks, we need to get the rest of the subjects over here. How do we do that? Well, the same place where the button that appeared before that said available masks was now shows go to main folder. Now, we're back here. So, the first", "math": "", "diagram": "" }, "related": [ "tutorial-ch40", "tutorial-ch42" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 4987, "chapterStart": "01:23:07" }, "images": [] }, { "id": "tutorial-ch42", "title": "Transfer segmentation from reference to all subjects", "keywords": [ "tutorial", "transcript", "video", "youtube", "transfer segmentation from reference to all subjects" ], "summary": "Use Transform masks → reference mask to subjects to warp the reference segmentation across the cohort (optional per-subject, label, AI redo, or threshold).", "options": [], "outputs": [], "transcriptSegments": [ { "text": "thing we want to do is we want to transfer the existing mask on the reference to all our subjects.", "startSeconds": 5135.36 }, { "text": "The way we do that is we go up here to this option that's called transform masks and then we can choose to only transfer it to a specific subject. We can also choose to only transfer a specific label. We can have the option to redo the AI process after it gets transferred and we can also choose to", "startSeconds": 5142.08 }, { "text": "apply a threshold after it's transferred. By default, I recommend to not touch any of these and just go ahead and click on reference mask to subjects.", "startSeconds": 5163.24 }, { "text": "Now, it reloaded. It finished transferring them. So, now we can go and do some quality checks.", "startSeconds": 5173.08 }, { "text": "So now you can see that by default it gives you this mask available for current edit, and now we can also see it in the refresh options. We can choose the mask label one. It doesn't look very smooth, and as we can see in the image", "startSeconds": 5179.2 }, { "text": "below, it probably is inaccurate. Now, this is one of the moments where you have to recognize that for tasks such as the segmentation of internal organs, the quality of the image impacts this a lot. And we knew this from the start. This subject had blurred out inside of the head, so it was", "startSeconds": 5194.4 }, { "text": "problematic in that regard, even though the outside may have looked pretty decent and still works for landmark", "startSeconds": 5215.88 } ], "algorithm": { "text": "thing we want to do is we want to transfer the existing mask on the reference to all our subjects. The way we do that is we go up here to this option that's called transform masks and then we can choose to only transfer it to a specific subject. We can also choose to only transfer a specific label. We can have the option to redo the AI process after it gets transferred and we can also choose to apply a threshold after it's transferred. By default, I recommend to not touch any of these and just go ahead and click on reference mask to subjects. Now, it reloaded. It finished transferring them. So, now we can go and do some quality checks. So now you can see that by default it gives you this mask available for current edit, and now we can also see it in the refresh options. We can choose the mask label one. It doesn't look very smooth, and as we can see in the image below, it probably is inaccurate. Now, this is one of the moments where you have to recognize that for tasks such as the segmentation of internal organs, the quality of the image impacts this a lot. And we knew this from the start. This subject had blurred out inside of the head, so it was problematic in that regard, even though the outside may have looked pretty decent and still works for landmark", "math": "", "diagram": "" }, "related": [ "tutorial-ch41", "tutorial-ch43" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5134, "chapterStart": "01:25:34" }, "images": [] }, { "id": "tutorial-ch43", "title": "Check the quality of the transferred segmentations", "keywords": [ "tutorial", "transcript", "video", "youtube", "check the quality of the transferred segmentations" ], "summary": "Visually QC transferred masks—internal contrast issues can degrade organ segmentations even when outer surfaces looked fine.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "placements. So now we load it, and we can take a look at the results, and we can definitely see issues specifically in the back of the head here. It seems like the top was very decent. The bottom has some issues, and the back has some issues. Now, the other one looked very", "startSeconds": 5224.08 }, { "text": "good on the inside, so it's less likely that this one will have big issues. So let's see what happens. Okay, so this is the mesh for that one, and this is the deformed brain. Let's see. So now", "startSeconds": 5241.8 }, { "text": "let's take a look at the full mask. Okay, so now we are looking at this one, and to me it looks like it did a phenomenal job. Great. So now what we want to do, disregard the fact that this image number 50 had that issue that we observed,", "startSeconds": 5253.72 } ], "algorithm": { "text": "placements. So now we load it, and we can take a look at the results, and we can definitely see issues specifically in the back of the head here. It seems like the top was very decent. The bottom has some issues, and the back has some issues. Now, the other one looked very good on the inside, so it's less likely that this one will have big issues. So let's see what happens. Okay, so this is the mesh for that one, and this is the deformed brain. Let's see. So now let's take a look at the full mask. Okay, so now we are looking at this one, and to me it looks like it did a phenomenal job. Great. So now what we want to do, disregard the fact that this image number 50 had that issue that we observed,", "math": "", "diagram": "" }, "related": [ "tutorial-ch42", "tutorial-ch44" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5224, "chapterStart": "01:27:04" }, "images": [] }, { "id": "tutorial-ch44", "title": "Apply the masks to all images", "keywords": [ "tutorial", "transcript", "video", "youtube", "apply the masks to all images" ], "summary": "Apply masks batch-wide for a chosen label and destination folder so every subject gets an isolated masked volume.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "we now will apply the masks to all of them. So what what we're going to do is go into apply masks, this option over here.", "startSeconds": 5273.24 }, { "text": "Select the label, which is label one, and then under mask folder, click on brain, and then apply all. So now that it finished, we go back here into mask available, into brain, and now we see the three of them.", "startSeconds": 5282.36 } ], "algorithm": { "text": "we now will apply the masks to all of them. So what what we're going to do is go into apply masks, this option over here. Select the label, which is label one, and then under mask folder, click on brain, and then apply all. So now that it finished, we go back here into mask available, into brain, and now we see the three of them.", "math": "", "diagram": "" }, "related": [ "tutorial-ch43", "tutorial-ch45" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5273, "chapterStart": "01:27:53" }, "images": [] }, { "id": "tutorial-ch45", "title": "Place landmarks on the segmented tissue", "keywords": [ "tutorial", "transcript", "video", "youtube", "place landmarks on the segmented tissue" ], "summary": "On the masked tissue, place landmarks, save, and snap to mesh so points sit on the segmented surface before transfer.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Okay, so now we go into the place guide points landmarks mode, then we click on landmarks, and now we can go ahead and start placing some landmarks.", "startSeconds": 5294.52 }, { "text": "And now that we're happy with it, we can just go ahead and click save. So now it's finished, and we can now check the performance of the landmarks. So we will go here and then check. Yeah, they seem to not be super well matched, but then we can just snap to mesh. Okay, now it's", "startSeconds": 5331.2 }, { "text": "snapped. We can go and check. Yeah, now it everything looks like it is at zero, and we can now go ahead and transfer the", "startSeconds": 5349.76 } ], "algorithm": { "text": "Okay, so now we go into the place guide points landmarks mode, then we click on landmarks, and now we can go ahead and start placing some landmarks. And now that we're happy with it, we can just go ahead and click save. So now it's finished, and we can now check the performance of the landmarks. So we will go here and then check. Yeah, they seem to not be super well matched, but then we can just snap to mesh. Okay, now it's snapped. We can go and check. Yeah, now it everything looks like it is at zero, and we can now go ahead and transfer the", "math": "", "diagram": "" }, "related": [ "tutorial-ch44", "tutorial-ch46" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5294, "chapterStart": "01:28:14" }, "images": [] }, { "id": "tutorial-ch46", "title": "Transfer the segmented tissue's landmarks to all subjects", "keywords": [ "tutorial", "transcript", "video", "youtube", "transfer the segmented tissue's landmarks to all subjects" ], "summary": "Propagate tissue landmarks to subjects, investigate snap outliers with right-click crosshair jumps, optionally redo AI after transfer, and export a tissue-specific CSV in the same space as surface landmarks.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "landmarks. Same way we did before, we go and click on snap landmarks to mesh, and then click on reference to subjects.", "startSeconds": 5358.6 }, { "text": "Now that it's finished, we can see that new values appear in the landmarking information. So we won't even go check the one that's problematic. I'm just going to go check the other one, even though it says it's snapped correctly 100% from 83%. It won't be representative of the actual anatomy of", "startSeconds": 5368.48 }, { "text": "that brain, so we shouldn't bother. So for the second one, we have all the landmarks placed in their correct location and snapped to the mesh. Then we can visualize how far they were snapping from. There is a strange flop over here that we see, and I wonder where that got sourced from, but it", "startSeconds": 5387.0 }, { "text": "seems like the landmarks snapped straight to it. One way to investigate things like this is we can go ahead and place our cursor on the area that we're interested in and then right click.", "startSeconds": 5408.56 }, { "text": "That will position the crosshairs down here in the image automatically in the place where we clicked.", "startSeconds": 5421.56 }, { "text": "So we can see that that flop actually does exist in the voxel image, which is this area that didn't get masked out.", "startSeconds": 5429.24 }, { "text": "However, it is not part of the brain. For issues like this one, you will need to redo it, clean it up manually, or toggling on the option of applying the AI after the transfer. Options like this could enhance situations where unwanted sections get filtered through the process. Finally, we can just export", "startSeconds": 5439.0 }, { "text": "this same landmarks here as a CSV. We can click that and you will see that it gets named different. Now it's brain landmarks export and now you can see the landmarks here are the ones for the brain and they are in the same exact space as the landmarks that we did for the outer surface of the head, which", "startSeconds": 5460.08 }, { "text": "means that you could combine them to do analysis of covariance and things like this. Finally, before I finish this", "startSeconds": 5480.28 } ], "algorithm": { "text": "landmarks. Same way we did before, we go and click on snap landmarks to mesh, and then click on reference to subjects. Now that it's finished, we can see that new values appear in the landmarking information. So we won't even go check the one that's problematic. I'm just going to go check the other one, even though it says it's snapped correctly 100% from 83%. It won't be representative of the actual anatomy of that brain, so we shouldn't bother. So for the second one, we have all the landmarks placed in their correct location and snapped to the mesh. Then we can visualize how far they were snapping from. There is a strange flop over here that we see, and I wonder where that got sourced from, but it seems like the landmarks snapped straight to it. One way to investigate things like this is we can go ahead and place our cursor on the area that we're interested in and then right click. That will position the crosshairs down here in the image automatically in the place where we clicked. So we can see that that flop actually does exist in the voxel image, which is this area that didn't get masked out. However, it is not part of the brain. For issues like this one, you will need to redo it, clean it up manually, or toggling on the option of applying the AI after the transfer. Options like this could enhance situations where unwanted sections get filtered through the process. Finally, we can just export this same landmarks here as a CSV. We can click that and you will see that it gets named different. Now it's brain landmarks export and now you can see the landmarks here are the ones for the brain and they are in the same exact space as the landmarks that we did for the outer surface of the head, which means that you could combine them to do analysis of covariance and things like this. Finally, before I finish this", "math": "", "diagram": "" }, "related": [ "tutorial-ch45" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5356, "chapterStart": "01:29:16" }, "images": [] } ] }, { "id": "tutorial-additional-tools", "title": "Additional tools", "methods": [ { "id": "tutorial-ch47", "title": "Denoise", "keywords": [ "tutorial", "transcript", "video", "youtube", "denoise" ], "summary": "Denoise with algorithm presets and live previews (optional auto-update); apply when satisfied, then clean mesh if filtering introduces outer-surface artifacts.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "tutorial, I would like to show an option that I didn't dive deep into, which is the denoise.", "startSeconds": 5489.28 }, { "text": "Now I quickly made this separate folder with a copy of one of the individuals. So what we're going to do here is we go up here into the options and we see denoise.", "startSeconds": 5496.28 }, { "text": "Then we have a drop-down menu with a lot of denoising algorithms. The most common one that you might already have heard about is Gaussian filter.", "startSeconds": 5506.36 }, { "text": "So one thing you can do with this is you can preview what this filter does. So if you toggle on denoise only current scan, it will show these two squares over here where we will see a preview. So, the way we do this is we put our crosshair somewhere where we want to see how the", "startSeconds": 5516.28 }, { "text": "noise looks like. So, let's say we want to look at this area. Now, we go back here and then we click on generate preview. And now here we can see the original one and then the denoised version using the Gaussian filter and the parameters that", "startSeconds": 5534.76 }, { "text": "we selected above here. Each algorithm will show you different parameters to set up. So, for example, the Gaussian filter here, it has a sigma and a truncate kernel radius. Now, if we want to see the preview automatically get updated the moment we change these parameters, we can toggle on the auto", "startSeconds": 5550.68 }, { "text": "mode down here. But, bear in mind that for this to work smoothly, you need to have a very powerful computer that can actually recalculate the preview very fast. So, we can try different previews for different algorithms. The non-local means, for example, has a 3D GPU option", "startSeconds": 5569.28 }, { "text": "and a 3D CPU option. Once you're happy with any of the results, you can click on the denoise button. And that's it, it just reloaded. Note that this denoising approach generated this outer surface bands that are unwanted. These ones you can get rid of just using the clean", "startSeconds": 5587.44 }, { "text": "mesh, but now there is much less noise.", "startSeconds": 5607.08 } ], "algorithm": { "text": "tutorial, I would like to show an option that I didn't dive deep into, which is the denoise. Now I quickly made this separate folder with a copy of one of the individuals. So what we're going to do here is we go up here into the options and we see denoise. Then we have a drop-down menu with a lot of denoising algorithms. The most common one that you might already have heard about is Gaussian filter. So one thing you can do with this is you can preview what this filter does. So if you toggle on denoise only current scan, it will show these two squares over here where we will see a preview. So, the way we do this is we put our crosshair somewhere where we want to see how the noise looks like. So, let's say we want to look at this area. Now, we go back here and then we click on generate preview. And now here we can see the original one and then the denoised version using the Gaussian filter and the parameters that we selected above here. Each algorithm will show you different parameters to set up. So, for example, the Gaussian filter here, it has a sigma and a truncate kernel radius. Now, if we want to see the preview automatically get updated the moment we change these parameters, we can toggle on the auto mode down here. But, bear in mind that for this to work smoothly, you need to have a very powerful computer that can actually recalculate the preview very fast. So, we can try different previews for different algorithms. The non-local means, for example, has a 3D GPU option and a 3D CPU option. Once you're happy with any of the results, you can click on the denoise button. And that's it, it just reloaded. Note that this denoising approach generated this outer surface bands that are unwanted. These ones you can get rid of just using the clean mesh, but now there is much less noise.", "math": "", "diagram": "" }, "related": [ "tutorial-ch48" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5486, "chapterStart": "01:31:26" }, "images": [] }, { "id": "tutorial-ch48", "title": "Farewell message", "keywords": [ "tutorial", "transcript", "video", "youtube", "farewell message" ], "summary": "Closing notes: contact the lab via the website for questions, test datasets, or feature requests so Aurora can better fit your workflow.", "options": [], "outputs": [], "transcriptSegments": [ { "text": "Okay, that was it. Thank you so much for um following this tutorial, and I hope your questions about the program were clarified here.", "startSeconds": 5611.2 }, { "text": "If you still have any remaining questions, you can email us through the website. If you have any specific data you would want to test, please let us know. We can help you out. Make sure it works flawlessly for your objectives. In the same way, if you have new features that you think would be very beneficial", "startSeconds": 5621.68 }, { "text": "for you, let us know and we will add them to our future developments.", "startSeconds": 5640.28 } ], "algorithm": { "text": "Okay, that was it. Thank you so much for um following this tutorial, and I hope your questions about the program were clarified here. If you still have any remaining questions, you can email us through the website. If you have any specific data you would want to test, please let us know. We can help you out. Make sure it works flawlessly for your objectives. In the same way, if you have new features that you think would be very beneficial for you, let us know and we will add them to our future developments.", "math": "", "diagram": "" }, "related": [ "tutorial-ch47" ], "video": { "youtubeId": "qWIXwDOJEUc", "startSeconds": 5611, "chapterStart": "01:33:31" }, "images": [] } ] } ] }, { "id": "project-scan-io", "title": "Project & Scan I/O", "subcategories": [ { "id": "file-browser", "title": "File Browser & Upload", "methods": [ { "id": "list-directories", "title": "List Directories", "keywords": [ "file browser", "directory listing", "filesystem", "browse", "extracted" ], "summary": "Lists child directories and files at a filesystem path for Aurora's project file browser. Expands tilde to the user home directory, supports a special root alias, and returns absolute path plus parallel file size arrays. When browsing inside an extracted folder, it flattens one level into subject subfolders so the UI can show scan contents without extra navigation.", "options": [ { "name": "path", "type": "string", "default": "~", "description": "URL-encoded directory path to list; use root for filesystem root." } ], "algorithm": { "text": "Expand and normalize the requested path, verify it is a directory, then enumerate non-hidden entries. Inside extracted/, list each subject folder's immediate children instead of top-level extracted entries. Return directories, files, sizes, absolute path, and missing flag when the path does not exist.", "math": "", "diagram": "flowchart TD\n A[\"GET path param\"] --> B[\"Expand ~ / root\"]\n B --> C{\"Path is directory?\"}\n C -->|\"no\"| D[\"Return missing=true\"]\n C -->|\"yes\"| E{\"basename == extracted?\"}\n E -->|\"yes\"| F[\"List each subject subfolder contents\"]\n E -->|\"no\"| G[\"List immediate children\"]\n F --> H[\"Return dirs, files, sizes, path\"]\n G --> H" }, "related": [ "quick-access-paths", "create-folder" ], "video": null, "images": [ { "caption": "Entries from multiple subject folders populate the extracted/ listing in one flattened browse response.", "path": "/static/docs-figures/docs_list_directories.gif" } ], "outputs": [ { "name": "directories", "type": "string[]", "description": "Child directory names (or flattened subject children when browsing extracted/)." }, { "name": "files", "type": "string[]", "description": "Child file names listed alongside directories." }, { "name": "sizes", "type": "number[]", "description": "Byte sizes parallel to files (same order)." }, { "name": "path", "type": "string", "description": "Absolute path that was listed." }, { "name": "missing", "type": "boolean", "description": "True when the requested path is not an existing directory." } ] }, { "id": "quick-access-paths", "title": "Quick Access Paths", "keywords": [ "file browser", "shortcuts", "home", "desktop", "drives", "OS paths" ], "summary": "Returns OS-specific shortcut paths for the file browser: home, desktop, documents, downloads, and mount or drive roots. On Windows it enumerates drive letters; on Linux and macOS it adds common mount points such as /media, /mnt, /Volumes, and /run/media/ when present.", "options": [], "algorithm": { "text": "Detect platform with platform.system(), build standard user folders from Path.home(), then append drive letters on Windows or known mount directories on Linux and macOS. Fail gracefully and still return home paths if enumeration errors occur.", "math": "", "diagram": "flowchart TD\n A[\"Detect OS\"] --> B[\"Add home/desktop/documents/downloads\"]\n B --> C{\"OS type?\"}\n C -->|\"Windows\"| D[\"psutil.disk_partitions drive letters\"]\n C -->|\"Darwin\"| E[\"/Volumes, /mnt if exist\"]\n C -->|\"Linux\"| F[\"/media, /mnt, /run/media/user\"]\n D --> G[\"Return quick_paths JSON\"]\n E --> G\n F --> G" }, "related": [ "list-directories" ], "video": null }, { "id": "create-folder", "title": "Create Folder", "keywords": [ "create", "folder" ], "summary": "Creates a new subdirectory under a path chosen in Aurora's file explorer. Use when organizing project folders (for example before importing scans) without leaving the UI.", "options": [ { "name": "path", "type": "string", "default": "required", "description": "Parent directory in which to create the folder." }, { "name": "folderName", "type": "string", "default": "required", "description": "Name of the new folder to create under path." } ], "algorithm": { "text": "1) Read path and folderName from the POST body. 2) Join them into a full filesystem path. 3) Call os.makedirs to create the directory (fails if it already exists or the parent is invalid). 4) Return a success message, or an error string with status 400 on exception.", "math": [], "diagram": "flowchart TD\n A[\"POST path, folderName\"] --> B[\"Join full path\"]\n B --> C[\"os.makedirs\"]\n C -->|\"ok\"| D[\"message: Folder created\"]\n C -->|\"error\"| E[\"error + HTTP 400\"]", "references": [] }, "related": [ "list-directories", "file-upload", "file-copy" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Success text when the folder is created." }, { "name": "error", "type": "string", "description": "Error detail when creation fails (HTTP 400)." } ] }, { "id": "file-upload", "title": "File Upload", "keywords": [ "file", "upload" ], "summary": "Uploads a multipart file into a chosen project folder so it becomes available on disk for extraction or import. Use when bringing external files into the open Aurora project from the file browser.", "options": [ { "name": "target_folder", "type": "string", "default": ".", "description": "Directory where the uploaded file is written; defaults to the current directory." }, { "name": "file", "type": "file", "default": "required", "description": "Multipart file field (request.FILES['file']) whose original name is used on disk." } ], "algorithm": { "text": "1) Read target_folder from the POST body (default '.'). 2) Take the uploaded multipart file from request.FILES['file']. 3) Write it chunk-by-chunk to target_folder/file.name. 4) Return a success message. Uses MultiPartParser and FormParser.", "math": [], "diagram": "flowchart TD\n A[\"POST multipart file\"] --> B[\"Resolve target_folder\"]\n B --> C[\"Write chunks to disk\"]\n C --> D[\"message: uploaded\"]", "references": [] }, "related": [ "create-folder", "file-copy", "list-directories" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Confirms the file was written successfully." } ] }, { "id": "file-copy", "title": "File Copy", "keywords": [ "file", "copy" ], "summary": "Copies one or more source folder trees into a target folder inside the managed project filesystem. Use when duplicating subject or dataset directories without re-uploading.", "options": [ { "name": "source_paths", "type": "string[]", "default": "required", "description": "Source path(s) resolved via find_absolute_paths / extract_folder_paths before copy." }, { "name": "target_folder", "type": "string", "default": "required", "description": "Destination parent directory for each copied tree." } ], "algorithm": { "text": "1) Resolve source_paths with find_absolute_paths and extract_folder_paths; read target_folder. 2) Reject the request if sources or target are missing, or if resolved sources are not a list. 3) For each source path, shutil.copytree into target_folder/. 4) Return success, or an error with HTTP 500 if copy fails.", "math": [], "diagram": "flowchart TD\n A[\"POST source_paths, target_folder\"] --> B[\"Resolve absolute folder paths\"]\n B --> C{\"Valid list + target?\"}\n C -->|\"no\"| D[\"HTTP 400 error\"]\n C -->|\"yes\"| E[\"copytree each source\"]\n E -->|\"ok\"| F[\"Directories copied\"]\n E -->|\"fail\"| G[\"HTTP 500 error\"]", "references": [] }, "related": [ "create-folder", "file-upload", "list-directories" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Success text when all directories copy." }, { "name": "error", "type": "string", "description": "Validation or copy failure detail (400/500)." } ] }, { "id": "get-data-size-directory", "title": "get Data Size Directory", "keywords": [ "get", "data", "size", "directory" ], "summary": "Reports filesystem capacity and the approximate on-disk size of a project directory in gigabytes. Use before large imports or batch jobs to see how much space the project already occupies; large trees may be deferred or skipped so the UI stays responsive.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project or folder path to measure; expanded with ~ and validated as an existing directory." } ], "algorithm": { "text": "1) Normalize directory to an absolute existing path. 2) If a cache entry younger than CACHE_TTL_SECONDS (30) exists, return it with cached=true. 3) If a scan is already in flight for this directory, return the previous cached result (deferred) or disk usage only with size=null and HTTP 202. 4) Otherwise mark the directory in-flight, read total/used disk via psutil.disk_usage, and walk files summing sizes until MAX_SCAN_SECONDS (2.5) elapses. 5) Set size and complete; cache a full result when the walk finishes; always clear the in-flight mark.", "math": [], "diagram": "flowchart TD\n A[\"POST directory\"] --> B[\"Normalize path\"]\n B --> C{\"Fresh cache?\"}\n C -->|\"yes\"| D[\"Return cached payload\"]\n C -->|\"no\"| E{\"Scan in flight?\"}\n E -->|\"yes\"| F[\"Return deferred / 202\"]\n E -->|\"no\"| G[\"Walk files up to 2.5s\"]\n G --> H[\"Return disk + size\"]", "references": [] }, "related": [ "list-directories", "clear-intermediate-files" ], "video": null, "outputs": [ { "name": "total_size", "type": "number", "description": "Total capacity of the filesystem containing the directory, in GB." }, { "name": "occupied_size", "type": "number", "description": "Used capacity of that filesystem, in GB." }, { "name": "size", "type": "number|null", "description": "Sum of file sizes under the directory in GB, or null if the timed walk was skipped." }, { "name": "complete", "type": "boolean", "description": "True when the directory walk finished within the time budget." }, { "name": "cached", "type": "boolean", "description": "True when the payload was served from the short-lived in-memory cache." }, { "name": "deferred", "type": "boolean", "description": "True when another scan for this directory is already in flight (may return HTTP 202)." }, { "name": "message", "type": "string", "description": "Human-readable note when the scan is deferred, timed out, or otherwise incomplete." }, { "name": "error", "type": "string", "description": "Validation or OS error detail (HTTP 400)." } ] } ] }, { "id": "filesystem", "title": "Project Files & Settings", "methods": [ { "id": "flag-subjects", "title": "Flag Subjects", "keywords": [ "flag", "subjects" ], "summary": "Sets or clears the flag field on one or more subject metadata JSON files under extracted/. Use when marking subjects so batch tools and the UI can include or exclude them consistently.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted//.json." }, { "name": "filenames", "type": "string|string[]", "default": "required", "description": "Subject folder name(s) whose metadata JSON should be updated; a single string is wrapped into a list." }, { "name": "flag", "type": "boolean", "default": true, "description": "Value written to metadata flag; coerced with bool()." } ], "algorithm": { "text": "1) Require directory and a non-empty filenames list (normalize a scalar to a list). 2) For each subject name, open extracted//.json, set flag to the boolean value, write indented JSON, and re-read to confirm. 3) Collect updated names, confirmed_flags, and per-file errors. 4) If nothing updated and errors exist, return HTTP 400; otherwise return the update summary.", "math": [], "diagram": "flowchart TD\n A[\"POST directory, filenames, flag\"] --> B[\"Normalize filenames list\"]\n B --> C[\"For each subject JSON\"]\n C --> D[\"Write flag + confirm\"]\n D --> E{\"Any updated?\"}\n E -->|\"no\"| F[\"HTTP 400 + details\"]\n E -->|\"yes\"| G[\"updated, confirmed_flags\"]", "references": [] }, "related": [ "report-bad-file", "list-directories" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Success text when at least one subject was updated." }, { "name": "updated", "type": "string[]", "description": "Subject names whose metadata was written successfully." }, { "name": "errors", "type": "object[]", "description": "Per-subject failures (invalid name, missing JSON, I/O/JSON errors)." }, { "name": "confirmed_flags", "type": "object", "description": "Map of subject name to the flag value re-read from disk after write." }, { "name": "error", "type": "string", "description": "Top-level error when no subjects could be updated." }, { "name": "details", "type": "object[]", "description": "Error details when the whole request fails with no updates." } ] }, { "id": "report-bad-file", "title": "Report Bad File", "keywords": [ "report", "bad", "file" ], "summary": "Marks a subject as faulty (or clears that mark) in its extracted metadata JSON. Use when a scan should be skipped or highlighted as broken in downstream batch processing.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted//.json." }, { "name": "filename", "type": "string", "default": "required", "description": "Subject folder name whose metadata JSON is updated." }, { "name": "faulty", "type": "boolean", "default": "required", "description": "Faulty status to store; coerced with bool(), or False when null." } ], "algorithm": { "text": "1) Build the path extracted//.json under directory. 2) Load the JSON, set faulty to bool(faulty) or False when faulty is null. 3) Write the metadata back with indent=4. 4) Return a success message.", "math": [], "diagram": "", "references": [] }, "related": [ "flag-subjects", "list-directories" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Confirms the faulty status was written." } ] }, { "id": "clear-intermediate-files", "title": "Clear Intermediate Files", "keywords": [ "cleanup", "storage", "elastic lock-in", "faulty delete", "intermediate edits" ], "summary": "Irreversibly cleans completed projects to save disk space. Deletes faulty subjects from extracted/ and moves their raw counterparts into directory/faulty/. For subjects with completed elastic registration, keeps only the elastic edit and the immediately prior edit plus originals, deletes other intermediate edits, and trims lossy_compression metadata to the last two entries. Emits WebSocket progress per subject.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted/ subjects." } ], "algorithm": { "text": "Iterate extracted subjects, delete entire folder when faulty (track raw names for faulty/ relocation), else find latest elastic edit number, retain elastic and edit N-1 files plus originals, delete other edits and auxiliary files, prune lossy_compression list, update JSON, report space freed.", "math": [ { "equation": "K = \\{E_{\\mathrm{elastic}},\\, E_{\\mathrm{elastic}}-1\\} \\cup \\mathrm{originals}", "caption": "K: files kept after cleanup; E_elastic: latest elastic-registration edit index; originals: unedited source files." } ], "diagram": "flowchart TD\n A[\"directory\"] --> B[\"For each extracted subject\"]\n B --> C{\"faulty?\"}\n C -->|\"yes\"| D[\"Delete folder + queue raw move\"]\n C -->|\"no\"| E{\"Has elastic edit?\"}\n E -->|\"no\"| F[\"Skip\"]\n E -->|\"yes\"| G[\"Delete intermediate edits\"]\n G --> H[\"Trim lossy_compression metadata\"]\n D --> I[\"Progress + summary\"]\n F --> I\n H --> I" }, "related": [ "report-bad-file", "create-atlas" ], "video": null }, { "id": "get-project-settings", "title": "Get Project Settings", "keywords": [ "get", "project", "settings" ], "summary": "Loads project-wide settings from extracted/project_settings.json for the open directory. Use when the UI needs the selected reference subject and general preferences after opening a project.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root; settings are read from extracted/project_settings.json." } ], "algorithm": { "text": "1) Resolve extracted/project_settings.json under directory. 2) If the file exists, load selected_reference and general_settings (defaults '' and {}). 3) If missing or unreadable, return empty selected_reference and empty general_settings. 4) Never invent other keys beyond what is stored in that JSON.", "math": [], "diagram": "", "references": [] }, "related": [ "save-project-settings", "get-label-names" ], "video": null, "outputs": [ { "name": "selected_reference", "type": "string", "description": "Saved reference subject name, or empty string if unset/missing/corrupt." }, { "name": "general_settings", "type": "object", "description": "Saved general settings object, or {} if unset/missing/corrupt." } ] }, { "id": "save-project-settings", "title": "Save Project Settings", "keywords": [ "save", "project", "settings" ], "summary": "Persists the selected reference and general settings into extracted/project_settings.json. Use when the user changes project-wide UI or processing preferences that should survive reloads.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root; creates extracted/ if needed." }, { "name": "selected_reference", "type": "string", "default": "", "description": "Reference subject name stored in project settings." }, { "name": "general_settings", "type": "object", "default": {}, "description": "Arbitrary general settings object written alongside the reference." } ], "algorithm": { "text": "1) Read directory, selected_reference (default ''), and general_settings (default {}). 2) Ensure the extracted/ parent exists. 3) Load existing project_settings.json or start with {}. 4) Overwrite selected_reference and general_settings, write indent=4 JSON, and return success or an error.", "math": [], "diagram": "flowchart TD\n A[\"POST settings fields\"] --> B[\"makedirs extracted/\"]\n B --> C[\"Merge into project_settings.json\"]\n C --> D[\"Write JSON\"]\n D --> E[\"message or error\"]", "references": [] }, "related": [ "get-project-settings", "save-label-names" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Success text when settings are written." }, { "name": "error", "type": "string", "description": "Failure detail (HTTP 400)." } ] }, { "id": "delete-edit", "title": "Delete Edit", "keywords": [ "delete", "edit" ], "summary": "Deletes a numbered edit slot (or the entire raw subject when edit is null) for a scan, including companion masks, projections, landmarks, and registration sidecars, then cleans related metadata. Use when rolling back a processing step or removing a bad edit from the timeline.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root for the subject (or atlas) being edited." }, { "name": "filename", "type": "string", "default": "required", "description": "Subject name (or atlas) whose edit files live under extracted// (atlas uses the atlas folder)." }, { "name": "edit", "type": "string|null", "default": "required", "description": "Edit slot index/string to delete; when null/absent path, deletes raw base volumes/mesh and metadata instead." } ], "algorithm": { "text": "1) Resolve scan_dir (atlas folder for filename atlas, else extracted/) and load metadata to detect preserved mesh (is_mesh true and voxelized false), falling back to PLY globs. 2) If edit is set: glob edit NIfTI/lossy/PLY payloads, projection PNGs, landmarks, elastic .mat, guidepoints, masks (.nii.mask.gz), and mesh .npy/.npz caches; delete them. 3) Update subject JSON: trim lossy_compression entries; clear alignment / elastic / backfixed / landmarks / histogram-match metadata (and restore threshold shifts) based on filename tags; for masked edits, optionally restore elastic backups and un-bump elastic files from edit+1 back to this slot. 4) If edit is null: remove base .nii.gz / lossy / .ply, base projections, JSON, and landmark files (and wipe atlas folder contents when applicable). 5) Return previous_edit (prior slot basename when edit > 0) with message Edit deleted, or 404 when no edit files matched.", "math": [], "diagram": "flowchart TD\n A[\"POST directory, filename, edit\"] --> B{\"edit set?\"}\n B -->|\"no\"| C[\"Delete raw + metadata\"]\n B -->|\"yes\"| D[\"Glob edit payloads + sidecars\"]\n D --> E{\"Files found?\"}\n E -->|\"no\"| F[\"404 Edit not found\"]\n E -->|\"yes\"| G[\"Delete files\"]\n G --> H[\"Clean JSON metadata\"]\n H --> I[\"Return previous_edit\"]\n C --> I", "references": [] }, "related": [ "projection-png", "overwrite-threshold", "clear-intermediate-files" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Edit deleted on success, or Edit not found when no matching edit files existed." }, { "name": "previous_edit", "type": "string|null", "description": "Basename of the prior edit file when edit > 0 and a previous slot exists; otherwise null." } ] } ] }, { "id": "extract-display", "title": "Extract & Display", "methods": [ { "id": "extract-scans-preflight", "title": "Extract all scans (preflight)", "keywords": [ "preflight", "mesh scale", "extraction preview", "scale groups", "voxelize" ], "summary": "Analyzes pending scans before extraction, especially mesh imports, to detect scale mismatches and recommend grouping. Returns mesh centroid statistics, auto-derived scale groups, and counts of mesh versus volume scans that still need extraction. Use this before Extract Scans when importing PLY/OBJ/STL subjects of uncertain physical scale.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing raw scan files." }, { "name": "scans", "type": "object[]", "default": null, "description": "Scan descriptors with at least name and subtype fields, same shape as extract-scans." } ], "algorithm": { "text": "Reuse ExtractScansView helpers to filter scans not yet fully extracted, split mesh and volume candidates, load mesh geometry summaries, cluster subjects by centroid size into scale groups, and return mesh_infos, scale_groups, subject_scale_groups, and has_scale_warning when multiple groups exist.", "math": [ { "equation": "\\mathrm{split\\ if}\\ \\mathrm{centroid\\_size} > 1.5 \\times \\mathrm{median}(\\mathrm{group})", "caption": "centroid_size: mesh scale proxy per subject; group: subjects currently clustered together; 1.5× median: scale-warning threshold." } ], "diagram": "flowchart TD\n A[\"scans list\"] --> B[\"Filter needs extraction\"]\n B --> C[\"Split mesh vs volume\"]\n C --> D[\"Load mesh summaries\"]\n D --> E[\"Group by centroid size\"]\n E --> F[\"Return scale warnings + counts\"]" }, "related": [ "extract-scans" ], "video": null }, { "id": "extract-scans", "title": "Extract all scans", "keywords": [ "import", "DICOM", "NIfTI", "MINC", "mesh", "voxelize", "segmentation", "lossy compression" ], "summary": "Imports raw scans from a project directory into extracted// as Aurora-ready NIfTI volumes or preserved PLY meshes. Supports DICOM folders, NIfTI, MINC, TIFF stacks, SCANCO AIM, and mesh formats (OBJ/PLY/STL/glTF/GLB). Creates full-resolution and lossy preview volumes, metadata JSON, optional segmentation masks, anisotropic resampling, and WebSocket progress updates. Skips already-extracted subjects, atlas entries, and standalone segmentation sidecars.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing raw scan files and folders." }, { "name": "scans", "type": "object[]", "default": null, "description": "Each entry requires name (string) and subtype (string, e.g. DICOM, NIfTI (.nii.gz), PLY Mesh)." }, { "name": "mesh_extraction_options", "type": "object", "default": "{}", "description": "Mesh import options: mode (keep|voxelize), skipped_subjects (string[]), subject_scale_factors ({name: factor}), subject_scale_groups ({name: group label})." } ], "algorithm": { "text": "Filter out fully extracted, segmentation-only, and atlas scans. For each remaining scan, load by subtype, standardize dtype, resample anisotropic volumes to isotropic voxels when needed, attach paired .seg masks if present, write .nii.gz plus lossy preview, threshold metadata, scanner header fields, and projection PNGs. Mesh keep mode saves welded PLY with metadata only; voxelize mode fills a bounded voxel grid then follows the volume pipeline. Emit WebSocket progress per scan.", "math": [ { "equation": "\\mathrm{voxel\\_size} \\ge \\max\\!\\left(\\frac{2\\cdot\\mathrm{mean}(\\mathrm{centroid\\_size})}{512},\\, \\frac{\\max(\\mathrm{extent})}{508}\\right)", "caption": "voxel_size: isotropic spacing written when voxelizing meshes; centroid_size / extent: subject size estimates from the mesh cohort; 512: MESH_MAX_VOXEL_CUBE; 508: usable_dim = MESH_MAX_VOXEL_CUBE − 2·MESH_VOXEL_PADDING with MESH_VOXEL_PADDING = 2 (2 voxels of padding on each side of the 512³ cube)." } ], "diagram": "flowchart TD\n A[\"scans + directory\"] --> B[\"Skip extracted/seg/atlas\"]\n B --> C{\"For each scan\"}\n C --> D{\"Subtype\"}\n D -->|\"Volume\"| E[\"Load + resample + save NIfTI\"]\n D -->|\"Mesh keep\"| F[\"Save PLY + metadata\"]\n D -->|\"Mesh voxelize\"| G[\"Voxelize → volume path\"]\n E --> H[\"Attach seg mask + lossy + JSON\"]\n F --> I[\"Progress update\"]\n G --> H\n H --> I" }, "related": [ "extract-scans-preflight", "extract-json", "get-grid-view" ], "video": null, "images": [ { "caption": "Raw import preview resolves into the first extracted surface stage.", "path": "/static/docs-figures/docs_extract_scans.gif" } ] }, { "id": "extract-json", "title": "Extract JSON Metadata", "keywords": [ "metadata", "edits list", "landmarks index", "scan json", "guidepoints" ], "summary": "Loads a subject metadata JSON file and enriches it with live filesystem state: sorted edit filenames, mask edit filenames, landmark file list, legacy six-voxel outlier counts, and whether guidepoints exist for the latest edit. Strips ephemeral persisted edit lists from disk JSON when migrating legacy metadata.", "options": [ { "name": "path", "type": "string", "default": null, "description": "Absolute path to a subject or atlas metadata JSON file." } ], "algorithm": { "text": "Read JSON, infer scan name, strip ephemeral edit lists from disk if present, enumerate edit and mask files from the subject folder, attach sorted edits/edits_masks/landmark_files, derive legacy outliers_six from distance files when missing, detect guidepoints on latest edit, and return the enriched metadata object.", "math": "", "diagram": "flowchart TD\n A[\"path to JSON\"] --> B[\"Load + strip ephemeral keys\"]\n B --> C[\"List edits/masks/landmarks on disk\"]\n C --> D[\"Derive legacy outlier counts\"]\n D --> E[\"Check guidepoints\"]\n E --> F[\"Return enriched JSON\"]" }, "related": [ "get-landmarks", "get-project-settings" ], "video": null }, { "id": "display-nifti", "title": "Display NIfTI", "keywords": [ "NIfTI download", "lossy volume", "full resolution", "slice viewer", "binary stream" ], "summary": "Serves a scan or edit NIfTI file as a binary download for the frontend slice viewer. By default returns the requested lossy or full-res file on disk; when request_full_res is true, loads the lossless volume, reapplies stored lossy compression parameters, and streams a freshly compressed preview without downsampling resolution.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Base or lossy NIfTI filename for the subject." }, { "name": "edit", "type": "string|null", "default": null, "description": "Specific edit filename; when set and containing edit, replaces filename." }, { "name": "request_full_res", "type": "boolean", "default": false, "description": "When true, rebuild compressed preview from lossless NIfTI using metadata lossy_compression settings." } ], "algorithm": { "text": "Resolve subject path under extracted/ or atlas/, optionally swap to edit filename, then either stream the on-disk file or load lossless NIfTI, read scale_factor/zero_point_shift/resolution_factor from metadata, recompress in memory, and return FileResponse octet-stream attachment.", "math": [ { "equation": "v_{\\mathrm{lossy}} = (v_{\\mathrm{raw}} - z) / s", "caption": "v_lossy: streamed 8-bit intensity; v_raw: full-resolution value; z: zero_point_shift; s: scale_factor from metadata." } ], "diagram": "flowchart TD\n A[\"directory + filename + edit\"] --> B{\"request_full_res?\"}\n B -->|\"no\"| C[\"Stream existing NIfTI file\"]\n B -->|\"yes\"| D[\"Load lossless + metadata params\"]\n D --> E[\"Recompress without shrink\"]\n E --> F[\"Stream compressed bytes\"]" }, "related": [ "nifti-header-info", "marchingcubes" ], "video": null, "images": [ { "caption": "Sequential mid-planes illustrate scrubbing a NIfTI volume in the viewer.", "path": "/static/docs-figures/docs_display_nifti.gif" } ] }, { "id": "nifti-header-info", "title": "Nifti Header Info", "keywords": [ "nifti", "header", "info" ], "summary": "Returns voxel dimensions and pixel spacing from a NIfTI header for a scan or edit. Use when the viewer or physical-unit tools need shape and zooms without loading the full volume array.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root used to locate extracted// or the atlas folder." }, { "name": "filename", "type": "string", "default": "required", "description": "Base scan filename; .nii.gz and _lossy are stripped to derive the subject name." }, { "name": "edit", "type": "string", "default": "optional", "description": "When provided and contains the substring edit, used as the on-disk filename instead of filename." } ], "algorithm": { "text": "1) Require directory and filename. 2) Derive subject name by stripping .nii.gz and _lossy; if edit contains edit, use edit as the file name. 3) Resolve the path under atlas/ or extracted// (lossy's _lossy removed from the path segment). 4) nib.load the file and return dims and pixDims from header shape and zooms (first three axes). 5) Return 404/500 with error when the file is missing or load fails.", "math": [], "diagram": "flowchart TD\n A[\"POST directory, filename, edit\"] --> B[\"Resolve NIfTI path\"]\n B --> C{\"File exists?\"}\n C -->|\"no\"| D[\"HTTP 404\"]\n C -->|\"yes\"| E[\"nib.load header\"]\n E --> F[\"dims + pixDims\"]", "references": [] }, "related": [ "display-nifti", "projection-png" ], "video": null, "outputs": [ { "name": "filename", "type": "string", "description": "Basename of the NIfTI file that was opened." }, { "name": "dims", "type": "number[]", "description": "First three voxel dimensions from the NIfTI header shape." }, { "name": "pixDims", "type": "number[]", "description": "First three pixel spacings (zooms) from the NIfTI header." }, { "name": "error", "type": "string", "description": "Missing fields, missing file, or load failure detail." } ] }, { "id": "projection-png", "title": "Projection Png", "keywords": [ "projection", "png" ], "summary": "Serves a cached isometric projection PNG for a scan edit (front or back) used by the grid and subject cards. Use when the UI needs a lightweight preview without loading the full volume or mesh.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Query param: project root (URL-decoded, ~ expanded)." }, { "name": "scan_name", "type": "string", "default": "required", "description": "Query param: subject name (or atlas); must stay under the project directory." }, { "name": "edit", "type": "string", "default": "required", "description": "Query param: edit volume/mesh filename used to derive projection candidate paths." }, { "name": "side", "type": "string", "default": "optional", "description": "Query param: when back, prefer *_projection_back.png; otherwise front *_projection.png." } ], "algorithm": { "text": "1) Require directory, scan_name, and edit query params; expand/unquote and reject paths outside the project. 2) Resolve scan_dir (atlas folder or extracted/). 3) Build candidate PNG paths from the edit name (PLY stem, lossy edit variants, or lossy projection suffixes); pick the first existing .png. 4) If none match, call resolve_mesh_projection_path. 5) Stream the PNG as image/png, or return 404 when missing.", "math": [], "diagram": "flowchart TD\n A[\"GET directory, scan_name, edit, side\"] --> B[\"Validate under project\"]\n B --> C[\"Try projection candidates\"]\n C --> D{\"PNG found?\"}\n D -->|\"no\"| E[\"resolve_mesh_projection_path\"]\n D -->|\"yes\"| F[\"FileResponse PNG\"]\n E --> F\n E -->|\"still none\"| G[\"HTTP 404\"]", "references": [] }, "related": [ "ensure-mesh-projection-pngs", "ensure-quick-grid-projections", "delete-edit" ], "video": null, "outputs": [ { "name": "(binary PNG)", "type": "image/png", "description": "FileResponse body of the first matching projection PNG on disk." }, { "name": "error", "type": "string", "description": "JSON error when params are missing, the path is invalid, or no PNG is found." } ] }, { "id": "physically-accurate-mesh", "title": "Physically Accurate Mesh Export", "keywords": [ "PLY export", "GLB export", "physical coordinates", "marching cubes", "download mesh" ], "summary": "Exports a subject mesh in physical millimeter coordinates for external tools. Voxel scans run full-resolution marching cubes at the stored threshold (optional Gaussian pre-blur), apply Aurora's X/Z axis convention, and write both GLB and ASCII PLY to the subject folder before streaming the PLY download. Preserved mesh subjects stream the native PLY edit directly without re-voxelization.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject folder name or atlas." }, { "name": "edit", "type": "string|null", "default": null, "description": "Edit filename; lossy suffix stripped for full-res source." }, { "name": "gaussian_blur", "type": "number", "default": 0.0, "description": "Gaussian sigma applied before marching cubes; clamped to 0–10." } ], "algorithm": { "text": "If preserved mesh metadata, resolve PLY edit path and stream binary PLY. Else load full-res NIfTI, optional gaussian_filter, marching_cubes at metadata threshold with physical spacing, swap X/Z axes and fix winding, save GLB and ASCII PLY beside subject, return PLY FileResponse attachment.", "math": [ { "equation": "V = \\mathrm{MC}(I,\\, \\tau,\\, \\Delta)", "caption": "V: mesh vertices; MC: marching cubes; I: volume; τ: iso-threshold from metadata; Δ: physical voxel spacing." } ], "diagram": "flowchart TD\n A[\"directory + filename + edit\"] --> B{\"Preserved mesh?\"}\n B -->|\"yes\"| C[\"Stream PLY edit\"]\n B -->|\"no\"| D[\"Load full-res volume\"]\n D --> E[\"Optional Gaussian blur\"]\n E --> F[\"Marching cubes at threshold\"]\n F --> G[\"Save GLB + PLY, stream PLY\"]" }, "related": [ "marchingcubes", "extract-scans" ], "video": null, "images": [ { "caption": "A volume mid-slice transitions into the marching-cubes surface exported with physical millimeter spacing.", "path": "/static/docs-figures/docs_physically_accurate_mesh.gif" } ] }, { "id": "get-all-subjects-mesh", "title": "Get All Subjects Mesh Overlay", "keywords": [ "overlay", "crop mode", "combined mesh", "OR mask", "preserved PLY", "reference frame" ], "summary": "Builds one combined GLB overlay of all non-faulty subjects aligned in the reference display frame for crop-mode show-all-scans. Voxel subjects contribute threshold masks resampled to a shared reference grid, unioned, largest-component filtered, and meshed with marching cubes; preserved PLY subjects load decimated native meshes in shared millimeters. Merges all parts, centers/scales like the reference Quick Mesh, and returns GLB plus overlay debug stats.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "reference", "type": "string", "default": null, "description": "Reference subject name defining the shared overlay coordinate frame." }, { "name": "full_resolution", "type": "boolean", "default": false, "description": "When true, use full-resolution reference voxel context for mask resampling." } ], "algorithm": { "text": "List valid non-faulty extracted subjects, resolve reference anchor and voxel grid, for each voxel subject load mask in reference space and keep largest connected component, OR-combine masks and marching-cubes the union, for each PLY subject load shared-mm mesh and decimate to per-subject budget, merge meshes, apply reference center/scale, build GLB via MarchingCubesView helper.", "math": [ { "equation": "M_{\\cup} = \\bigvee_i M_i", "caption": "M_∪: combined foreground mask; M_i: largest connected component of subject i in reference space; ∨: voxel-wise OR." }, { "equation": "S = \\mathrm{MC}(M_{\\cup})", "caption": "S: overlay surface; MC: marching cubes on the union mask." } ], "diagram": "flowchart TD\n A[\"reference + directory\"] --> B[\"Split voxel vs PLY subjects\"]\n B --> C[\"Voxel: resample masks → OR → MC\"]\n B --> D[\"PLY: load shared-mm meshes\"]\n C --> E[\"Merge + reference center/scale\"]\n D --> E\n E --> F[\"Return GLB + overlay_debug\"]" }, "related": [ "marchingcubes", "apply-crop" ], "video": null } ] }, { "id": "volume-edits", "title": "Volume Edits", "methods": [ { "id": "apply-rotation", "title": "Apply Rotation", "keywords": [ "rotate", "3D rotation", "affine", "batch rotate", "reorientation" ], "summary": "Applies a 3D rotation from the Quick Mesh Canvas to a subject NIfTI edit or to all scans in a project. Converts the BabylonJS rotation matrix to scipy affine_transform coordinates, rotates around the mesh center stored in metadata, uses tight bounding-box sizing to limit memory, saves a new *_edit_* rotated volume plus lossy copy, and propagates masks and landmarks.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject name for single-scan rotation; reference context when all is true." }, { "name": "edit", "type": "string|null", "default": null, "description": "Source edit filename; null uses base .nii.gz." }, { "name": "rotation", "type": "object", "default": null, "description": "BabylonJS rotation payload with _m indexed matrix entries remapped to 3×3 numpy rotation." }, { "name": "interpolation", "type": "string", "default": "linear", "description": "nearest (order 0) or linear (order 1) interpolation during affine resampling." }, { "name": "all", "type": "boolean", "default": false, "description": "When true, apply the same rotation to all eligible scans via rotate_all_scans." } ], "algorithm": { "text": "Build rotation_matrix from rotation._m, load full-res NIfTI and metadata center converted to voxel ZYX, estimate background, compute the minimal rotated axis-aligned bounding box, affine_transform the volume about center (linear or nearest-neighbor for labels), keep a border of 20 background voxels on all faces to reduce later registration edge artifacts, save new edit and lossy companion, copy/transform paired masks and landmarks.", "math": [ { "equation": "v' = R(v - c) + c + o", "caption": "v: input voxel; R: rotation; c: rotation center (voxel ZYX); o: optional offset. Output canvas includes a 20-voxel background border on each face." } ], "diagram": "flowchart TD\n A[\"rotation matrix + edit\"] --> B{\"all scans?\"}\n B -->|\"yes\"| C[\"rotate_all_scans\"]\n B -->|\"no\"| D[\"Load volume + mesh center\"]\n D --> E[\"Compute tight rotated bbox\"]\n E --> F[\"affine_transform\"]\n F --> G[\"Save edit + lossy + propagate masks/LM\"]" }, "related": [ "apply-crop", "marchingcubes" ], "video": null, "images": [ { "caption": "The specimen orientation updates from the pre-rotation state to the rotated pose.", "path": "/static/docs-figures/docs_apply_rotation.gif" } ] }, { "id": "apply-crop", "title": "Apply Crop", "keywords": [ "crop", "bounding box", "padding", "edit", "mask propagation", "landmarks" ], "summary": "Crops a subject volume to a 3D bounding box from the UI, adds configurable padding filled with the estimated background value, and saves a new *_edit_*_cropped NIfTI plus matching lossy edit. Propagates paired masks and landmarks with identical crop/pad geometry, filtering landmarks outside the crop.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject folder name." }, { "name": "edit", "type": "string|null", "default": null, "description": "Source edit filename; null uses base full-res NIfTI." }, { "name": "crop", "type": "number[6]", "default": null, "description": "Axis-aligned bounds [x_min, y_min, z_min, x_max, y_max, z_max] in voxel order mapped to ZYX slices with ±2 voxel margin." }, { "name": "padding", "type": "number", "default": 40, "description": "Voxels of background padding added on every face after cropping." } ], "algorithm": { "text": "Load full-res NIfTI (strip _lossy from edit), map crop box to ZYX indices with small margin, slice volume, pad with background_value, save cropped edit and lossy derivative, mirror crop/pad on .nii.mask.gz if present with downsampled lossy mask, transform landmarks and filter distance arrays to kept indices.", "math": [ { "equation": "V_{\\mathrm{crop}} = \\mathrm{pad}\\!\\left(V[z_0{:}z_1,\\, y_0{:}y_1,\\, x_0{:}x_1],\\, p,\\, b\\right)", "caption": "V: volume; [z0:z1, …]: crop box in ZYX; p: padding width; b: background fill value." } ], "diagram": "flowchart TD\n A[\"crop box + edit\"] --> B[\"Load full-res volume\"]\n B --> C[\"Slice ZYX with margin\"]\n C --> D[\"Pad with background\"]\n D --> E[\"Save cropped edit + lossy\"]\n E --> F[\"Propagate mask + landmarks\"]" }, "related": [ "apply-rotation", "get-all-subjects-mesh" ], "video": null, "images": [ { "caption": "The surface FOV tightens from the full aligned mesh to the cropped shell.", "path": "/static/docs-figures/docs_apply_crop.gif" } ] }, { "id": "overwrite-threshold", "title": "Overwrite Threshold", "keywords": [ "overwrite", "threshold" ], "summary": "Writes a new integer threshold into a subject's (or atlas) metadata JSON. Use when meshing or foreground tools should pick a manually chosen intensity cutoff instead of the previous stored value.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root used to locate the subject or atlas JSON." }, { "name": "filename", "type": "string", "default": "required", "description": "Subject name, or atlas for the atlas JSON path." }, { "name": "threshold", "type": "number", "default": "required", "description": "New threshold stored as int(threshold) in metadata." } ], "algorithm": { "text": "1) Resolve .json under atlas/ or extracted//. 2) Load the JSON metadata. 3) Set threshold to int(threshold). 4) Write the file back with indent=4 and return HTTP 200 with an empty body.", "math": [], "diagram": "", "references": [] }, "related": [ "save-threshold-values", "apply-threshold" ], "video": null, "outputs": [] }, { "id": "get-label-names", "title": "Get Label Names", "keywords": [ "get", "label", "names" ], "summary": "Returns the project-wide map of label IDs to custom display names from extracted/project_settings.json. Use when multi-label masks need human-readable names shared across every subject in the project.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root passed to load_project_label_names." } ], "algorithm": { "text": "1) Read directory from the POST body. 2) Call load_project_label_names(directory). 3) On any exception, use {}. 4) Return { label_names }.", "math": [], "diagram": "", "references": [] }, "related": [ "save-label-names", "get-project-settings", "mask-out-labels" ], "video": null, "outputs": [ { "name": "label_names", "type": "object", "description": "Map of label id string to custom name; {} if loading fails." } ] }, { "id": "save-label-names", "title": "Save Label Names", "keywords": [ "save", "label", "names" ], "summary": "Creates or updates a single label's custom display name in the project-wide label map (or clears it when the name is blank). Use when renaming segmentation labels so every scan in the project shows the same names.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root; mapping is stored in extracted/project_settings.json." }, { "name": "label", "type": "string|number", "default": "required", "description": "Label id to rename; required." }, { "name": "name", "type": "string", "default": "", "description": "Custom display name; blank removes the custom mapping so the UI falls back to Label ." } ], "algorithm": { "text": "1) Require label in the POST body; read directory and name (default ''). 2) Call save_project_label_name(directory, label, name), which persists into project_settings.json. 3) Return the updated label_names map, or HTTP 400 with error on failure.", "math": [], "diagram": "", "references": [] }, "related": [ "get-label-names", "save-project-settings", "mask-out-labels" ], "video": null, "outputs": [ { "name": "label_names", "type": "object", "description": "Updated full label-name map after the save." }, { "name": "error", "type": "string", "description": "Validation or save failure detail (HTTP 400)." } ] } ] }, { "id": "landmarks-atlas", "title": "Landmarks & Atlas", "methods": [ { "id": "random-landmarking", "title": "Random Landmarking", "keywords": [ "random landmarks", "Poisson disk", "FPS", "surface landmarks", "symmetry" ], "summary": "Automatically places a requested number of approximately evenly spaced landmarks on a subject surface. Voxel scans marching-cube the edit mesh; preserved meshes load PLY directly. Uses Poisson-disk sampling plus farthest-point refinement, optional symmetry mirroring on mesh projects, snaps landmarks to the surface, writes landmarks and distance JSON, and updates metadata landmark statistics.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject name or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Lossy edit filename; _lossy suffix stripped to locate full-res source." }, { "name": "num_landmarks", "type": "number", "default": null, "description": "Target count of landmarks to generate on the surface." } ], "algorithm": { "text": "Load mesh from preserved PLY or marching cubes on NIfTI edit, attempt symmetry-plane mirrored landmark warp on mesh projects else Poisson-disk oversample + farthest-point sample to num_landmarks, snap to surface, compute residual and snap distances, save _landmarks.json and _landmark_distances.json, write landmarks summary into subject metadata.", "math": [ { "equation": "\\max_{\\mathcal{S}} \\min_{i \\neq j \\in \\mathcal{S}} \\|x_i - x_j\\|", "caption": "S: selected landmark subset (FPS); x_i: candidate points from Poisson-disk sampling; objective: maximize nearest-neighbor separation." } ], "diagram": "flowchart TD\n A[\"num_landmarks + edit\"] --> B[\"Build surface mesh\"]\n B --> C{\"Symmetric mesh project?\"}\n C -->|\"yes\"| D[\"Mirror-warp landmarks\"]\n C -->|\"no\"| E[\"Poisson + FPS sample\"]\n D --> F[\"Snap to mesh + save JSON\"]\n E --> F" }, "related": [ "get-landmarks", "upload-landmarks" ], "video": null, "images": [ { "caption": "Automatically distributed surface landmarks provide broad geometric coverage.", "path": "/static/docs-figures/docs_random_landmarking.gif" } ] }, { "id": "get-landmarks", "title": "Get Landmarks", "keywords": [ "landmarks", "distances", "edit landmarks", "atlas landmarks" ], "summary": "Loads landmark coordinates and optional distance arrays for a subject edit. Resolves the correct landmarks JSON stem from the edit name, normalizes list or dict on-disk formats into indexed landmarks with position and landmark_type, and returns paired distances or false when no distance file exists.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject name or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Edit filename used to resolve _landmarks.json." } ], "algorithm": { "text": "Resolve sub_path and landmark edit stem, load landmarks JSON or 422 if missing, convert list entries to indexed dict with landmark_type, load matching _landmark_distances.json when present, return {landmarks, distances}.", "math": "", "diagram": "flowchart TD\n A[\"filename + edit\"] --> B[\"Resolve landmarks path\"]\n B --> C{\"File exists?\"}\n C -->|\"no\"| D[\"422 not found\"]\n C -->|\"yes\"| E[\"Normalize format\"]\n E --> F[\"Load distances if any\"]\n F --> G[\"Return payload\"]" }, "related": [ "upload-landmarks", "export-landmarks" ], "video": null }, { "id": "upload-landmarks", "title": "Upload Landmarks", "keywords": [ "import landmarks", "validate", "fit", "snap to mesh", "coordinate mismatch" ], "summary": "Imports external landmark coordinates for a subject edit with validate or fit actions. Validates mean mesh distance against a voxel-size tolerance, optionally runs RANSAC/mesh ICP fit with scaling and snap-to-mesh, then saves landmarks bundle JSON and updates metadata outlier statistics.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject name or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Edit filename associated with the uploaded landmarks." }, { "name": "landmarks", "type": "object", "default": null, "description": "Indexed landmark dict with position arrays (and optional landmark_type) from the client." }, { "name": "action", "type": "string", "default": "validate", "description": "validate checks mesh alignment; fit aligns landmarks to current mesh before save." }, { "name": "allow_scaling", "type": "boolean", "default": false, "description": "When action is fit, permit uniform scaling during landmark-to-mesh alignment." }, { "name": "snap_to_mesh", "type": "boolean", "default": false, "description": "When action is fit, snap fitted landmarks onto the mesh surface." }, { "name": "use_outer_shell", "type": "boolean", "default": true, "description": "Prefer outer-shell mesh for fit when combined with snap_to_mesh." } ], "algorithm": { "text": "Load mesh for edit, convert landmarks dict to array, optional fit via landmark_upload_aligner with scaling/snap/outer shell, compute mesh distances and outlier counts, reject with 409/422 on mismatch thresholds, else _save_landmarks_bundle and return success.", "math": [ { "equation": "\\mathrm{mismatch\\ if}\\ \\overline{d} > k \\cdot \\Delta", "caption": "d̄: mean landmark-to-surface distance after upload; Δ: subject voxel_size; k = LANDMARK_UPLOAD_MISMATCH_VOXEL_MULTIPLIER = 3 (mismatch when d̄ > 3·Δ)." } ], "diagram": "flowchart TD\n A[\"landmarks + action\"] --> B[\"Load mesh + metadata\"]\n B --> C{\"action == fit?\"}\n C -->|\"yes\"| D[\"ICP/RANSAC fit + optional snap\"]\n C -->|\"no\"| E[\"Validate distances only\"]\n D --> F{\"Within tolerance?\"}\n E --> F\n F -->|\"no\"| G[\"409/422 mismatch\"]\n F -->|\"yes\"| H[\"Save landmarks bundle\"]" }, "related": [ "snap-to-mesh", "alpaca-align-landmarks-to-mesh", "get-landmarks" ], "video": null }, { "id": "transfer-landmarks", "title": "Transfer landmarks", "keywords": [ "landmark transfer", "atlas to subjects", "deformation field", "propagation", "snap" ], "summary": "Propagates landmarks from reference or atlas onto subjects using elastic deformation fields: trilinear interpolation of the 8-neighborhood displacement, optional outer-shell snap, and stored mesh distances for QC. Supports reference→subjects, atlas→subjects (via mean inverse then per-subject forward), and reference→atlas modes.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "reference", "type": "string", "default": null, "description": "Reference subject name driving transfer geometry." }, { "name": "transferType", "type": "string", "default": null, "description": "Transfer mode: atlas-to-subjects, reference-to-subjects, or reference-to-atlas." }, { "name": "autoSnapToMesh", "type": "boolean", "default": null, "description": "When true, snap transferred landmarks onto target subject mesh surfaces." }, { "name": "prefer_mask", "type": "boolean", "default": false, "description": "Use segmentation mask geometry when resolving transfer targets." }, { "name": "mask_label", "type": "number", "default": 1, "description": "Label id used when prefer_mask is true." }, { "name": "overwriteExistingLandmarks", "type": "boolean", "default": true, "description": "Replace existing subject landmark files when true; skip subjects that already have landmarks when false." }, { "name": "use_outer_shell", "type": "boolean", "default": true, "description": "Use outer-shell mesh extraction when snapping transferred landmarks." } ], "algorithm": { "text": "1) Branch on transferType to choose source landmarks and target list; load atlas/reference transforms and fields. 2) Reference→subject: x_{S_i} = x_R + ũ_i(x_R) with ũ_i the forward field trilinearly sampled at x_R (8-voxel neighborhood). 3) Atlas→subject: compose φ_i(φ̄⁻¹(x_A)); reference→atlas uses φ̄(x_R). 4) Optionally snap to the target outer shell (shrink-factor aware for lossy fields), write landmarks + distance JSON, update metadata, skip faulty scans.", "math": [ { "equation": "\\mathbf{x}_{S_i} = \\phi_i(\\mathbf{x}_R) = \\mathbf{x}_R + \\tilde{\\mathbf{u}}_i(\\mathbf{x}_R)", "caption": "Reference→subject transfer; φ_i forward field on the reference grid; ũ_i(x_R): trilinear interpolation of the discrete displacement u_i at (generally off-grid) landmark x_R." }, { "equation": "\\mathbf{x}_{S_i} = \\phi_i\\!\\bigl(\\bar{\\phi}^{-1}(\\mathbf{x}_A)\\bigr)", "caption": "Atlas→subject: map atlas landmark x_A to reference by the mean inverse field, then to subject i by φ_i." } ], "diagram": "flowchart TD\n A[\"Source landmarks\"] --> B{\"transferType\"}\n B -->|ref→subjects| C[\"φ_i at x_R\"]\n B -->|atlas→subjects| D[\"φ̄⁻¹ then φ_i\"]\n B -->|ref→atlas| E[\"φ̄ at x_R\"]\n C --> F[\"Optional snap + distances\"]\n D --> F\n E --> F" }, "related": [ "create-atlas", "get-landmarks", "upload-landmarks" ], "video": null, "images": [ { "caption": "Landmark dots are carried onto the elastically registered specimen surface.", "path": "/static/docs-figures/docs_transfer_landmarks.gif" } ] }, { "id": "snap-to-mesh", "title": "Snap To Mesh", "keywords": [ "snap", "to", "mesh" ], "summary": "Projects a subject's landmarks onto the iso-surface (or outer shell) extracted from the current edit. Use after placing or transferring landmarks that sit slightly off the mesh so surface distances and morphometrics stay meaningful.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject id under extracted/, or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Edit basename (lossy suffix stripped). Defaults to {filename}.nii.gz." }, { "name": "prefer_mask", "type": "boolean", "default": false, "description": "When true and a paired .nii.mask.gz exists, snap against the mask==mask_label surface instead of intensity threshold." }, { "name": "mask_label", "type": "number", "default": 1, "description": "Mask label used when prefer_mask is true." }, { "name": "use_outer_shell", "type": "boolean", "default": true, "description": "When true, ALPACA get_outer_mesh keeps the visible outer surface before snapping." } ], "algorithm": { "text": "1) Resolve subject path and strip _lossy from edit. 2) Load the edit NIfTI and metadata threshold (or replace the volume with mask==mask_label at threshold 0.5 when prefer_mask). 3) Gaussian-smooth with σ = 0.3, run marching cubes at voxel_size spacing, optionally extract the outer shell. 4) Load {edit}_landmarks.json (preserving landmark_type), snap each position to the closest surface point, and warn when any move exceeds 2·voxel_size (snapping still proceeds). 5) Write updated landmarks, recompute surface/snap distance stats (outliers beyond 1· and 6· voxel_size), and update metadata.landmarks.", "math": [ { "equation": "p' = \\arg\\min_{q \\in S} \\|p - q\\|_2", "caption": "p: landmark; S: marching-cubes (optionally outer-shell) surface; p': snapped position. Warn when ‖p'−p‖ > 2·voxel_size." } ], "diagram": "flowchart TD\n A[\"Load edit + metadata\"] --> B{\"prefer_mask?\"}\n B -->|yes| C[\"Use mask label surface\"]\n B -->|no| D[\"Intensity threshold\"]\n C --> E[\"Marching cubes σ=0.3\"]\n D --> E\n E --> F{\"use_outer_shell?\"}\n F -->|yes| G[\"Outer shell (get_outer_mesh)\"]\n F -->|no| H[\"Full MC surface\"]\n G --> I[\"Closest-point snap\"]\n H --> I\n I --> J[\"Save landmarks + distances\"]", "references": [] }, "related": [ "get-landmarks", "upload-landmarks", "export-landmark-distances", "alpaca-get-outer-mesh" ], "video": null, "images": [ { "caption": "Landmark guesses settle onto the mesh shell after snapping.", "path": "/static/docs-figures/docs_snap_to_mesh.gif" } ], "outputs": [ { "name": "message", "type": "string", "description": "Success confirmation after landmarks and distance metadata are written." } ] }, { "id": "save-guidepoints", "title": "Save Guidepoints", "keywords": [ "save", "guidepoints" ], "summary": "Persists user-placed guidepoints for a subject edit as {edit}_guidepoints.json (or deletes the file when an empty list is sent). Guidepoints steer landmark curves and mesh-elastic initialization in the Aurora UI.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject id or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Edit stem; .nii.gz/.ply/_lossy stripped when present." }, { "name": "guidepoints", "type": "array", "default": null, "description": "List of {position:{x,y,z}}. Empty list deletes the guidepoints file." } ], "algorithm": { "text": "1) Normalize edit stem and resolve atlas vs extracted/ path. 2) If guidepoints is empty, delete the matching *_guidepoints.json and return. 3) Convert UI objects to [x,y,z] rows. 4) Write JSON beside the edit (or {filename}_guidepoints.json when edit is null). Note: a mesh-distance snap helper exists on the view but is not invoked on the current save path — points are stored as provided.", "math": [], "diagram": "flowchart TD\n A[\"Receive guidepoints\"] --> B{\"empty list?\"}\n B -->|yes| C[\"Delete *_guidepoints.json\"]\n B -->|no| D[\"Write [x,y,z] JSON\"]", "references": [] }, "related": [ "load-guidepoints", "get-semi-landmarks", "mesh-elastic-registration" ], "video": null, "outputs": [ { "name": "message", "type": "string", "description": "Saved or deleted confirmation." } ] }, { "id": "load-guidepoints", "title": "Load Guidepoints", "keywords": [ "load", "guidepoints" ], "summary": "Loads guidepoints for the requested edit, falling back through earlier edits when the current edit has none. Returns UI-shaped {index, position} objects and optionally previous_edit_guidepoints for restore prompts.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject id or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Edit stem, or latest to resolve the highest edit number." } ], "algorithm": { "text": "1) Normalize edit; if edit is latest, pick the highest *_edit_N_* volume/mesh. 2) Prefer {edit}_guidepoints.json, else {filename}_guidepoints.json. 3) Convert stored [x,y,z] rows to {index, position}. 4) If still empty and an edit number is known, search edit N−1 … 0 for *_guidepoints.json, then the unversioned file, and attach previous_edit_guidepoints when found.", "math": [], "diagram": "flowchart TD\n A[\"Resolve edit / latest\"] --> B[\"Try current guidepoints file\"]\n B -->|found| C[\"Return guidepoints\"]\n B -->|empty| D[\"Search previous edits\"]\n D --> E[\"Optional previous_edit_guidepoints\"]", "references": [] }, "related": [ "save-guidepoints", "get-semi-landmarks" ], "video": null, "outputs": [ { "name": "guidepoints", "type": "array", "description": "[{index, position:{x,y,z}}, ...] for the current edit (may be empty)." }, { "name": "previous_edit_guidepoints", "type": "object", "description": "Optional {source_edit, guidepoints} when the current edit has none but an older edit or unversioned file does." }, { "name": "message", "type": "string", "description": "Status message." } ] }, { "id": "get-semi-landmarks", "title": "Get Semi Landmarks", "keywords": [ "get", "semi", "landmarks" ], "summary": "Places sliding semi-landmarks between two mesh endpoints on the outer shell: construct a cutting plane from the chord’s residual PCA, Dijkstra-shortest contour path, arc-length equalization, then snap to the shell (Gunz & Mitteroecker–ready ordered samples).", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject id or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Lossy edit basename or latest." }, { "name": "start_coordinates", "type": "object", "default": null, "description": "{x,y,z} start landmark in physical space." }, { "name": "end_coordinates", "type": "object", "default": null, "description": "{x,y,z} end landmark in physical space." }, { "name": "num_semilandmarks", "type": "number", "default": null, "description": "Count of semi-landmarks placed strictly between the endpoints." } ], "algorithm": { "text": "1) Build the outer shell from the lossy NIfTI (σ = 0.3 Gaussian, marching cubes, shared Outer shell extraction (get_outer_mesh), largest component). 2) Sample 100 points on the straight start→end chord; snap each to the shell; remove the component along unit line direction d; take v_1 = first principal component of residuals; set plane normal n = d × v_1. 3) Keep outer-shell vertices within half the specimen voxel spacing of the plane. 4) Dijkstra (scipy.sparse.csgraph) on the plane contour graph for the shortest start→end path. 5) Place num_semilandmarks at equal arc length along that path; project to the plane; snap to shell; return interior points only.", "math": [ { "equation": "\\mathbf{n} = \\mathbf{d} \\times \\mathbf{v}_1", "caption": "Cutting-plane normal; d: unit start→end direction; v_1: first PC of shell residuals after removing the d component (from 100 chord samples)." }, { "equation": "t_i = \\frac{i}{n+1}\\, L,\\quad i=1..n", "caption": "Arc-length targets along the Dijkstra plane–shell path of length L; n = num_semilandmarks." } ], "diagram": "flowchart TD\n A[\"Outer shell from lossy NIfTI\"] --> B[\"Straight chord samples\"]\n B --> C[\"Cutting plane\"]\n C --> D[\"Dijkstra plane contour path\"]\n D --> E[\"Arc-length equalize\"]\n E --> F[\"Project + snap to shell\"]", "references": [] }, "related": [ "rebalance-landmarks-along-path", "alpaca-get-outer-mesh", "snap-to-mesh", "save-guidepoints" ], "video": null, "images": [ { "caption": "Semi-landmarks are placed along the cranial shell at approximately equal arc length.", "path": "/static/docs-figures/docs_get_semi_landmarks.gif" } ], "outputs": [ { "name": "semilandmarks", "type": "array", "description": "[{x,y,z}, ...] equalized snapped positions (excludes endpoints)." }, { "name": "message", "type": "string", "description": "Success summary with count." } ] }, { "id": "rebalance-landmarks-along-path", "title": "Rebalance Landmarks Along Path", "keywords": [ "rebalance", "landmarks", "along", "path" ], "summary": "Redistributes three or more ordered curve landmarks to equal arc-length spacing along an interpolating cubic spline through their current positions; endpoints stay fixed and middle points snap to the outer shell (paper §2.6.5b).", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject id or atlas." }, { "name": "edit", "type": "string", "default": null, "description": "Lossy edit used to build the outer shell context." }, { "name": "landmarks", "type": "array", "default": null, "description": "Ordered list of {index,x,y,z}; requires ≥ 3 points." } ], "algorithm": { "text": "1) Validate ≥ 3 landmarks each with index,x,y,z; resolve endpoint order. 2) Reuse GetSemiLandmarksView._load_shell_mesh_context for the outer shell. 3) Fit an interpolating cubic B-spline through all current positions (shape unchanged). 4) Densely sample the spline (max(500, n·200) samples), place points at equal arc-length fractions including endpoints, snap only the middle points to the shell, and return updated coordinates with original indices.", "math": [ { "equation": "p_i = \\gamma\\!\\left(\\frac{i}{n-1}\\right),\\quad i=0..n-1", "caption": "γ: interpolating spline through the input landmarks; equal arc-length parameter. Endpoints i=0 and i=n−1 stay put; interiors are snapped to the outer shell." } ], "diagram": "flowchart TD\n A[\"Ordered landmarks ≥3\"] --> B[\"Interpolating B-spline\"]\n B --> C[\"Equal arc-length samples\"]\n C --> D[\"Snap middles to outer shell\"]\n D --> E[\"Return same indices\"]", "references": [] }, "related": [ "get-semi-landmarks", "snap-to-mesh", "save-guidepoints" ], "video": null, "images": [ { "caption": "Landmark spacing along a path is equalized to uniform arc length.", "path": "/static/docs-figures/docs_rebalance_landmarks_along_path.gif" } ], "outputs": [ { "name": "landmarks", "type": "array", "description": "Same indices with updated x,y,z after rebalance + shell snap." }, { "name": "message", "type": "string", "description": "Count of rebalanced landmarks." } ] }, { "id": "export-landmarks", "title": "Export landmarks (file)", "keywords": [ "export", "landmarks" ], "summary": "Downloads a project-wide CSV of the latest landmark coordinates per non-faulty subject (Subject plus landmark_*_x/y/z columns, with semi_ prefix for semi-landmarks). One file for morphometric analysis outside Aurora.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root containing extracted/." } ], "algorithm": { "text": "1) List extracted/ subjects; skip missing metadata or faulty=true. 2) For each remaining subject, take the newest *landmarks.json by mtime. 3) Normalize list/dict formats; prefix semi_ when landmark_type is semi. 4) Build a wide table Subject, landmark_i_x/y/z; write a timestamped CSV under .temp/ and return it as a FileResponse.", "math": [], "diagram": "flowchart TD\n A[\"Scan extracted/\"] --> B[\"Skip faulty\"]\n B --> C[\"Latest *_landmarks.json\"]\n C --> D[\"Wide CSV rows\"]\n D --> E[\"FileResponse\"]", "references": [] }, "related": [ "export-landmarks-bundle", "export-landmarks-json", "export-landmark-distances", "get-landmarks" ], "video": null, "outputs": [ { "name": "(file)", "type": "text/csv", "description": "Attachment landmarks_export_YYYYMMDD_HHMMSS.csv." } ] }, { "id": "export-landmark-distances", "title": "Export landmark distances", "keywords": [ "export", "ZIP", "surface distances", "snap distances", "QC" ], "summary": "Exports per-landmark mesh distance QC data as a ZIP containing surface-distance CSV and, when available, pre-snap distance CSV. Surface distances reflect landmark-to-mesh residuals; snap distances record how far landmarks moved during snapping.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." } ], "algorithm": { "text": "For each non-faulty subject with landmarks, load paired _landmark_distances.json, extract distances and optional snap_distance arrays, build aligned CSV rows with semi_landmark naming, zip surface and snap CSVs into timestamped archive, return ZIP download.", "math": "", "diagram": "flowchart TD\n A[\"directory\"] --> B[\"Collect distance arrays per subject\"]\n B --> C[\"Build surface CSV\"]\n C --> D{\"snap_distance present?\"}\n D -->|\"yes\"| E[\"Add snap CSV to ZIP\"]\n D -->|\"no\"| F[\"ZIP surface CSV only\"]\n E --> G[\"Download ZIP\"]\n F --> G" }, "related": [ "export-landmarks-bundle", "get-landmarks" ], "video": null }, { "id": "export-landmarks-bundle", "title": "Export landmarks", "keywords": [ "export", "landmarks", "bundle" ], "summary": "Downloads a ZIP with coordinates CSV plus surface (and snap) distance CSVs for all non-faulty subjects that have landmarks — the one-click package matching separate coordinate and distance exporters.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root containing extracted/." } ], "algorithm": { "text": "1) Same subject filter as ExportLandmarksView. 2) Collect coordinate rows from each latest landmarks file (semi_ naming). 3) When a paired *_landmark_distances.json exists, add surface distance rows and snap_distance rows when present. 4) Zip the CSVs under a timestamped name in .temp/ and return as application/zip.", "math": [], "diagram": "flowchart TD\n A[\"Per-subject landmarks\"] --> B[\"Coordinates CSV\"]\n A --> C[\"Optional distance CSVs\"]\n B --> D[\"ZIP FileResponse\"]\n C --> D", "references": [] }, "related": [ "export-landmarks", "export-landmarks-json", "export-landmark-distances", "get-landmarks" ], "video": null, "outputs": [ { "name": "(file)", "type": "application/zip", "description": "ZIP with coordinates CSV and optional surface/snap distance CSVs." } ] }, { "id": "export-landmarks-json", "title": "Export landmarks (JSON)", "keywords": [ "export", "landmarks", "json" ], "summary": "Downloads a single JSON object mapping each non-faulty subject to its latest landmark dictionary (landmark_i / semi_landmark_i → [x,y,z]) for interchange with other tools.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root containing extracted/." } ], "algorithm": { "text": "1) Sort extracted/ subjects; skip faulty or missing metadata. 2) Load newest *landmarks.json per subject; normalize to a dict with semi_ prefixes when needed. 3) Write {scan_name: {landmark_key: [x,y,z]}} to .temp/ and return as a JSON FileResponse.", "math": [], "diagram": "", "references": [] }, "related": [ "export-landmarks", "export-landmarks-bundle", "get-landmarks" ], "video": null, "outputs": [ { "name": "(file)", "type": "application/json", "description": "Attachment {project}_landmarks_YYYYMMDD_HHMMSS.json." } ] }, { "id": "create-atlas", "title": "Create atlas", "keywords": [ "atlas", "population average", "elastic registration", "inverse transform", "ANTs" ], "summary": "Constructs a population atlas from hub-and-spoke elastic registrations without additional pairwise warps: average forward displacements on the reference grid define atlas geometry, and the mean of intensities pulled into reference space is warped by the inverse mean field. Optional post-step after elastic registration when a population-average target is desired.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory with extracted/ subjects." }, { "name": "reference", "type": "string", "default": null, "description": "Reference subject name whose space defines the atlas construction frame." } ], "algorithm": { "text": "1) Include the reference as identity (u_0 ≡ 0) with the N−1 movers that have elastic_fwd fields; skip faulty subjects. 2) Accumulate displacement vectors u_i on the shared reference grid and form the mean field φ̄(x) = x + (1/N) Σ u_i(x). 3) For each specimen, sample I_i(φ_i(x)) into reference space and average to Ī_ref(x). 4) Numerically invert φ̄ (ANTs invert_displacement_field) and set I_atlas(y) = Ī_ref(φ̄⁻¹(y)). 5) Write atlas intensities, fields, and metadata under atlas/.", "math": [ { "equation": "\\bar{\\phi}(\\mathbf{x}) = \\mathbf{x} + \\frac{1}{N}\\sum_{i=0}^{N-1}\\mathbf{u}_i(\\mathbf{x})", "caption": "Mean forward map on the reference grid; u_i: forward displacement of specimen i (u_0 ≡ 0 for the reference); N: cohort size including the reference." }, { "equation": "\\bar{I}_{\\mathrm{ref}}(\\mathbf{x}) = \\frac{1}{N}\\sum_{i=0}^{N-1} I_i\\!\\bigl(\\phi_i(\\mathbf{x})\\bigr)", "caption": "Mean intensity in reference geometry; φ_i(x) = x + u_i(x) pulls subject i into the reference frame." }, { "equation": "I_{\\mathrm{atlas}}(\\mathbf{y}) = \\bar{I}_{\\mathrm{ref}}\\!\\bigl(\\bar{\\phi}^{-1}(\\mathbf{y})\\bigr)", "caption": "Atlas intensity at physical coordinate y; φ̄⁻¹ is the numerical inverse of the mean forward field (ANTs invert_displacement_field on the shared grid)." } ], "diagram": "flowchart TD\n A[\"Forward fields φ_i on reference grid\"] --> B[\"Mean displacement φ̄\"]\n A --> C[\"Pull intensities → Ī_ref\"]\n B --> D[\"Invert φ̄\"]\n C --> E[\"I_atlas = Ī_ref(φ̄⁻¹)\"]\n D --> E" }, "related": [ "make-atlas-reference", "align-to-reference", "get-all-subjects-mesh" ], "video": null, "images": [ { "caption": "Population atlas and an elastically registered specimen shown in the common anatomical frame.", "path": "/static/docs-figures/docs_create_atlas.gif" } ] }, { "id": "make-atlas-reference", "title": "Make Atlas Reference", "keywords": [ "atlas reference", "ReferenceAtlas", "promote atlas", "selected reference" ], "summary": "Promotes the population atlas into a normal extracted reference subject (ReferenceAtlas): moves atlas/ or legacy extracted/atlas/ into extracted/ReferenceAtlas/, renames atlas-prefixed files, updates project_settings selected_reference, and removes obsolete average-only files. Idempotent when already promoted.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing atlas output and extracted/ folder." } ], "algorithm": { "text": "If already promoted, refresh settings via finalize_promoted_reference_atlas. Else locate atlas source folder, shutil.move to extracted/ReferenceAtlas, rename files from atlas* to ReferenceAtlas*, set selected_reference in project_settings.json, return paths and cleanup summary.", "math": "", "diagram": "flowchart TD\n A[\"directory\"] --> B{\"Already promoted?\"}\n B -->|\"yes\"| C[\"Refresh settings\"]\n B -->|\"no\"| D[\"Move atlas → extracted/ReferenceAtlas\"]\n D --> E[\"Rename files + update settings\"]\n C --> F[\"Return subject_path + selected_reference\"]\n E --> F" }, "related": [ "create-atlas", "get-project-settings" ], "video": null, "images": [ { "caption": "Population atlas is promoted from atlas/ into extracted/ReferenceAtlas/ as the project reference subject.", "path": "/static/docs-figures/docs_make_atlas_reference.gif" } ] } ] }, { "id": "volume-display", "title": "Volume Display & Previews", "methods": [ { "id": "ensure-quick-grid-projections", "title": "Ensure Quick Grid Projections", "keywords": [ "Quick Grid", "batch projection", "generate PNG", "non-elastic edit", "mesh projection" ], "summary": "Batch-generates missing front/back projection PNG pairs for Quick Grid subjects. For each requested scan, resolves the latest non-elastic NIfTI or preserved PLY edit, generates isometric PNGs from lossy or full volume data when absent, or ensures mesh projection PNGs for PLY edits.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "subjects", "type": "object[]", "default": null, "description": "Each item: scan_name (required), edit (optional hint; latest non-elastic edit is resolved when omitted)." } ], "algorithm": { "text": "For each subject, resolve latest non-elastic edit via lossy edit, edit glob, original NIfTI, or projection PNG hints. If front/back PNG pair exists, mark ready; else generate from lossy/full NIfTI with uint8 normalization or call ensure_mesh_projection_pngs for PLY. Return per-subject ready, generated, and debug fields.", "math": [], "diagram": "flowchart TD\n A[\"subjects list\"] --> B[\"Resolve latest non-elastic edit\"]\n B --> C{\"Projection pair exists?\"}\n C -->|\"yes\"| D[\"ready=true\"]\n C -->|\"no\"| E{\"PLY or NIfTI?\"}\n E -->|\"PLY\"| F[\"ensure_mesh_projection_pngs\"]\n E -->|\"NIfTI\"| G[\"generate_lossy_isometric_projection_png\"]\n F --> H[\"Append result\"]\n G --> H\n D --> H" }, "related": [ "projection-png", "get-grid-view" ], "video": null, "images": [ { "caption": "Quick-grid tiles fill as projection thumbnails become available.", "path": "/static/docs-figures/docs_ensure_quick_grid_projections.gif" } ] }, { "id": "get-grid-view", "title": "Get Grid View Preview", "keywords": [ "grid preview", "thumbnail", "histogram", "quadrant render", "preserved mesh" ], "summary": "Returns a base64 JPEG grid preview and intensity histogram for a subject or atlas, regenerating the thumbnail when the latest non-elastic edit or threshold changed. Voxel scans render four orthogonal/max-projection quadrants from the latest lossy edit; preserved PLY meshes render a mesh-specific preview JPEG.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "scan_name", "type": "string", "default": null, "description": "Subject folder name or atlas." }, { "name": "ignoreElastic", "type": "boolean", "default": false, "description": "When true, ignore elastic edits when picking the latest edit for preview staleness checks." } ], "algorithm": { "text": "Compare preview.json last_edit, threshold, and timestamp against latest edit file. If current, read cached preview.jpg; otherwise load latest lossy NIfTI or preserved mesh PLY, render quadrant JPEG or mesh preview, update preview.json, compute histogram from metadata/threshold, and return base64 image plus histogram_data.", "math": "", "diagram": "flowchart TD\n A[\"scan_name + directory\"] --> B{\"Preview up to date?\"}\n B -->|\"yes\"| C[\"Read preview.jpg\"]\n B -->|\"no\"| D{\"Preserved mesh?\"}\n D -->|\"yes\"| E[\"Render mesh preview\"]\n D -->|\"no\"| F[\"Render lossy quadrant JPEG\"]\n C --> G[\"Attach histogram\"]\n E --> G\n F --> G" }, "related": [ "projection-png", "ensure-quick-grid-projections", "extract-scans" ], "video": null, "images": [ { "caption": "A compact grid compares representative processing states across the project.", "path": "/static/docs-figures/docs_get_grid_view.gif" } ] } ] }, { "id": "mesh-generation", "title": "Mesh Generation & Overlay", "methods": [ { "id": "marchingcubes", "title": "Marching Cubes Mesh", "keywords": [ "marching cubes", "GLB", "Quick Mesh", "threshold", "heatmap", "preserved mesh", "shell mode" ], "summary": "Generates the interactive 3D mesh shown in Quick Mesh Canvas. Voxel scans run marching cubes on a lossy or full-res NIfTI at the project threshold (with optional Gaussian blur, shell mode, and label masks); preserved PLY scans load and decimate native mesh geometry. Returns base64 GLB plus center/scale_factor for BabylonJS display, optional base64-encoded per-vertex heatmap distances after elastic registration, and multi-label GLB when segmentations are present.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory." }, { "name": "filename", "type": "string", "default": null, "description": "Subject folder name or atlas." }, { "name": "edit", "type": "string|null", "default": null, "description": "Lossy edit filename, latest, explicit PLY edit, or null for base lossy/original volume." }, { "name": "threshold", "type": "number|string", "default": "fromfile", "description": "Iso-surface level; fromfile reads metadata threshold mapped through lossy compression params." }, { "name": "high_quality", "type": "boolean", "default": false, "description": "Use full voxel spacing instead of lossy resolution factor when computing mesh spacing." }, { "name": "shell_mode", "type": "boolean", "default": false, "description": "Extract outer shell surface instead of full solid isosurface." }, { "name": "full_resolution", "type": "boolean", "default": false, "description": "Load full-resolution NIfTI instead of lossy edit." }, { "name": "calculate_dense_distances", "type": "boolean", "default": false, "description": "Compute per-vertex registration heatmap distances for elastic edits." }, { "name": "gaussian_blur", "type": "number|null", "default": null, "description": "Gaussian sigma applied to volume before isosurface extraction." }, { "name": "mask_label", "type": "number|null", "default": null, "description": "When set with use_label_mask, restrict mesh to voxels matching this label id." }, { "name": "use_label_mask", "type": "boolean", "default": true, "description": "When true and mask_label set, use label mask instead of global threshold." }, { "name": "heatmap_metric", "type": "string", "default": "surface_distance", "description": "Registration heatmap metric id; legacy elastic_metric alias accepted. Falls back when unsupported for mesh/voxel combo." } ], "algorithm": { "text": "Route PLY or preserved-mesh metadata to decimated GLB loader with optional reference-surface distances. Otherwise load NIfTI, swap axes for BabylonJS, map threshold through lossy params, reject degenerate foreground ratios, apply optional blur/mask/shell, run marching cubes at spacing = voxel_size × resolution_factor, decimate to vertex budget, center and scale vertices, encode GLB and optional float32 distance buffer as base64.", "math": [ { "equation": "S = \\{ v \\mid I(v) \\ge \\tau \\}", "caption": "S: iso-surface; I(v): volume intensity at voxel v; τ: threshold (mapped through lossy params when needed)." }, { "equation": "\\Delta = \\delta \\cdot r", "caption": "Δ: marching-cubes spacing; δ: native voxel_size; r: resolution_factor from lossy compression metadata." } ], "diagram": "flowchart TD\n A[\"POST mesh request\"] --> B{\"PLY / is_mesh?\"}\n B -->|\"yes\"| C[\"Load/decimate preserved mesh GLB\"]\n B -->|\"no\"| D[\"Load NIfTI + map threshold\"]\n D --> E{\"Valid foreground ratio?\"}\n E -->|\"no\"| F[\"400 threshold error\"]\n E -->|\"yes\"| G[\"marching cubes + decimate\"]\n G --> H[\"Center/scale + GLB base64\"]\n C --> H\n H --> I{\"calculate_dense_distances?\"}\n I -->|\"yes\"| J[\"Encode heatmap distances\"]\n I -->|\"no\"| K[\"Return payload\"]" }, "related": [ "physically-accurate-mesh", "get-all-subjects-mesh", "overwrite-threshold" ], "video": null, "images": [ { "caption": "A volume mid-slice transitions into the extracted marching-cubes surface.", "path": "/static/docs-figures/docs_marchingcubes.gif" } ] } ] } ] }, { "id": "preprocessing", "title": "Preprocessing", "subcategories": [ { "id": "mesh-cleanup", "title": "Mesh Cleanup", "methods": [ { "id": "cleanup-mesh", "title": "Cleanup mesh", "keywords": [ "mesh", "islands", "connected components", "artifact removal", "gaussian blur", "threshold" ], "summary": "Removes spurious disconnected foreground islands from a single voxel mesh or atlas volume by thresholding, connected-component labeling, and resetting removed voxels to the estimated background intensity. Useful after segmentation or thresholding when stray blobs, noise islands, or partial connections remain; optional Gaussian pre-smoothing and volume-based island filtering refine results. Rejected when the source edit is from elastic registration.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted or atlas scan folders." }, { "name": "filename", "type": "string", "default": null, "description": "Scan or atlas base name; threshold is read from its metadata JSON." }, { "name": "edit", "type": "string|null", "default": null, "description": "Source NIfTI edit filename to clean; null uses the base scan file." }, { "name": "gaussian_blur", "type": "number|null", "default": null, "description": "Gaussian sigma applied before island detection; 0 or null disables smoothing and post-noise cleanup." }, { "name": "use_island_volume_threshold", "type": "boolean", "default": false, "description": "When true, keep all islands above a minimum volume fraction instead of keeping only the largest component." }, { "name": "min_island_volume_percent", "type": "number", "default": 1.0, "description": "Minimum island size as a percentage of total foreground voxels; clamped to 0.1–30.0." } ], "algorithm": { "text": "Load the latest or specified NIfTI edit and read threshold from metadata. Optionally Gaussian-smooth a copy for island detection. Estimate background as the mode (or mean fallback) of subsampled voxels below threshold. Label connected components where smoothed data exceeds threshold. In default mode, remove every component except the largest by setting those voxels to the background value. In volume-threshold mode, remove components smaller than min_island_volume_percent of foreground. If Gaussian blur was used, also remove voxels above threshold in the original data that fall below threshold in the smoothed volume. Save a new *_cleaned edit plus lossy copy, and propagate masks and landmarks.", "math": [ { "equation": "b = \\mathrm{mode}\\{ v \\mid v < \\tau \\}", "caption": "b: estimated background intensity; v: subsampled voxel values; τ: threshold from metadata." }, { "equation": "\\mathcal{I} = \\{ v \\mid (G_\\sigma * I)(v) > \\tau \\}", "caption": "I: island mask; G_σ: optional Gaussian smooth; σ: blur scale; τ: threshold." }, { "equation": "\\mathrm{keep} = \\arg\\max_C |C|\\ \\mathrm{or}\\ \\{ C : |C| \\ge p\\% \\cdot |F| \\}", "caption": "C: connected component; |C|: component size; F: foreground; p = min_island_volume_percent (default 1.0, clamped to 0.1–30) in volume-threshold mode." }, { "equation": "v_{\\mathrm{removed}} \\leftarrow b", "caption": "Removed island voxels are replaced with background b." } ], "diagram": "flowchart TD\n A[\"Load NIfTI + threshold\"] --> B{\"Elastic edit?\"}\n B -->|\"yes\"| X[\"Reject request\"]\n B -->|\"no\"| C[\"Optional Gaussian blur\"]\n C --> D[\"Estimate background below threshold\"]\n D --> E[\"Label connected components\"]\n E --> F{\"Multiple islands?\"}\n F -->|\"no\"| G[\"No cleanup\"]\n F -->|\"yes\"| H{\"Volume threshold mode?\"}\n H -->|\"no\"| I[\"Keep largest island\"]\n H -->|\"yes\"| J[\"Keep islands ≥ min volume %\"]\n I --> K[\"Set removed voxels to background\"]\n J --> K\n K --> L{\"Gaussian used?\"}\n L -->|\"yes\"| M[\"Remove blur-only noise voxels\"]\n L -->|\"no\"| N[\"Save cleaned edit + lossy\"]\n M --> N" }, "related": [ "batch-cleanup-mesh", "apply-threshold", "homogenize-background", "apply-mesh-cleanup" ], "video": null, "images": [ { "caption": "Cleanup removes peripheral signal between the aligned and cleaned volumes.", "path": "/static/docs-figures/docs_cleanup_mesh.gif" } ] }, { "id": "batch-cleanup-mesh", "title": "Batch cleanup mesh", "keywords": [ "batch", "mesh cleanup", "project-wide", "preserved mesh", "voxel mesh", "islands" ], "summary": "Runs mesh cleanup across all eligible scans in a project directory. Voxel-based scans use the same island-removal logic as Cleanup Mesh; preserved PLY meshes are routed to the preserved-mesh cleanup path. Skips elastic edits, already-cleaned edits, scans with no removable islands, and scans filtered by flag settings. Returns per-scan success, skip, or error results.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory whose extracted subfolders are processed." }, { "name": "gaussian_blur", "type": "number|null", "default": null, "description": "Gaussian sigma forwarded to voxel mesh cleanup; null disables blur." }, { "name": "use_island_volume_threshold", "type": "boolean", "default": false, "description": "When true, keep islands above the minimum volume fraction instead of only the largest island." }, { "name": "min_island_volume_percent", "type": "number", "default": 1.0, "description": "Minimum retained island volume as a percentage of foreground; clamped to 0.1–30.0." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter: off, exclude flagged scans, or only flagged scans." } ], "algorithm": { "text": "Enumerate non-faulty extracted scans, drop voxel scans with elastic registration, apply flag filter, then iterate. For preserved meshes, delegate to ApplyMeshCleanupView with latest edit. For voxel scans, load latest non-elastic, non-cleaned edit, run CleanupMeshView.clean_mesh, and save only when islands were actually removed. Emit WebSocket progress updates and aggregate results.", "math": "", "diagram": "flowchart TD\n A[\"List extracted scans\"] --> B[\"Apply flag filter\"]\n B --> C{\"For each scan\"}\n C --> D{\"Preserved PLY mesh?\"}\n D -->|\"yes\"| E[\"Preserved mesh cleanup\"]\n D -->|\"no\"| F{\"Elastic or already cleaned?\"}\n F -->|\"yes\"| G[\"Skip\"]\n F -->|\"no\"| H[\"Run clean_mesh\"]\n H --> I{\"Islands removed?\"}\n I -->|\"no\"| G\n I -->|\"yes\"| J[\"Save cleaned edit + lossy\"]\n E --> K[\"Collect result\"]\n G --> K\n J --> K" }, "related": [ "cleanup-mesh" ], "video": null } ] }, { "id": "background", "title": "Background", "methods": [ { "id": "homogenize-background", "title": "Background Offset Correction", "keywords": [ "background", "backfixed", "intensity offset", "batch", "threshold shift", "Set Background Value" ], "summary": "Background Offset Correction estimates a per-scan (or shared) background level—manually or as the mode of voxels below threshold—and subtracts it so backgrounds harmonize near zero (values below the estimate clip to the dtype floor), then shifts stored thresholds by the same offset.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "background_value", "type": "number|null", "default": null, "description": "Fixed background intensity to subtract from all scans; null triggers per-scan automatic estimation." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter: off, exclude flagged scans, or only flagged scans." } ], "algorithm": { "text": "Collect eligible voxel scans and optionally run a first pass that estimates each scan's background as the mode of subsampled voxels below threshold (dtype-specific histogram logic for uint8 and uint16). Subtract the chosen background from every voxel with clipping to dtype bounds, save *_backfixed edits and lossy copies, subtract the same amount from metadata threshold, record backfixed_shift, and copy masks and landmarks unchanged.", "math": [ { "equation": "b = \\mathrm{mode}\\{ v \\mid 0 < v < \\tau \\}", "caption": "b: per-scan (or shared) background estimate; v: voxel intensity; τ: current threshold." }, { "equation": "v' = \\mathrm{clip}(v - b)", "caption": "v': background-corrected intensity, clipped to the dtype range." }, { "equation": "\\tau' = \\tau - b", "caption": "τ': threshold shifted by the same background offset so segmentation stays consistent." } ], "diagram": "flowchart TD\n A[\"List eligible voxel scans\"] --> B{\"background_value provided?\"}\n B -->|\"no\"| C[\"Phase 1: estimate background per scan\"]\n B -->|\"yes\"| D[\"Use fixed background for all\"]\n C --> E[\"Phase 2: subtract background\"]\n D --> E\n E --> F[\"Clip to dtype range\"]\n F --> G[\"Save backfixed edit + lossy\"]\n G --> H[\"Update threshold and backfixed_shift\"]" }, "related": [ "save-background-values", "remove-background", "apply-threshold" ], "video": null, "images": [ { "caption": "Background is flattened from cleaned anatomy into the backfixed state.", "path": "/static/docs-figures/docs_homogenize_background.gif" } ] }, { "id": "remove-background", "title": "Remove Background", "keywords": [ "background removal", "backremoved", "minimum value", "batch", "threshold" ], "summary": "Sets all voxels below each scan's metadata threshold to that scan's global minimum intensity, producing a flattened background without changing geometry. Processes all eligible voxel scans except the reference scan and scans already backremoved or elastically registered. Useful when a uniform low background is desired for visualization or downstream intensity handling while preserving foreground values.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "reference", "type": "string|null", "default": null, "description": "Reference scan name excluded from processing." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter: off, exclude flagged scans, or only flagged scans." } ], "algorithm": { "text": "For each eligible scan, load the latest edit, read threshold from metadata, compute the volume minimum, assign min_value to all voxels strictly below threshold, save a new *_backremoved edit and lossy copy, and copy paired masks and landmarks without geometric change.", "math": [ { "equation": "v' = \\begin{cases} \\min(I) & v < \\tau \\\\ v & \\mathrm{otherwise} \\end{cases}", "caption": "v: input intensity; τ: threshold; values below τ are floored to the volume minimum (effectively cleared)." } ], "diagram": "flowchart TD\n A[\"List voxel scans except reference\"] --> B[\"Skip backremoved / elastic scans\"]\n B --> C[\"Apply flag filter\"]\n C --> D[\"Load latest edit + threshold\"]\n D --> E[\"min_value = min('volume')\"]\n E --> F[\"Set v < threshold to min_value\"]\n F --> G[\"Save backremoved edit + lossy\"]" }, "related": [ "homogenize-background", "apply-threshold" ], "video": null, "images": [ { "caption": "Aligned anatomy with background becomes the cleaned specimen without background.", "path": "/static/docs-figures/docs_remove_background.gif" } ] }, { "id": "save-background-values", "title": "Save Background Values", "keywords": [ "background", "manual correction", "backfixed", "per-scan", "UI values" ], "summary": "Applies user-specified per-scan background offsets from the UI, subtracting each value from the corresponding volume and updating metadata thresholds. Scans with background 0 are skipped. Each corrected scan is saved as a new *_backfixed edit with lossy copy, adjusted threshold, and backfixed_shift. Useful when automatic background estimation needs manual refinement for individual subjects.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "values", "type": "object", "default": {}, "description": "Map of scan_name to objects containing a numeric background field to subtract." } ], "algorithm": { "text": "Filter values to scans that include a background field and skip zero backgrounds. For each scan, load the latest edit, subtract the provided background using HomogenizeBackgroundView._homogenize_background, save *_backfixed output, reduce metadata threshold by the same amount, store backfixed_shift, generate lossy data, and copy masks and landmarks.", "math": [ { "equation": "v' = \\mathrm{clip}(v - b_s)", "caption": "b_s: saved per-scan background; v': corrected intensity clipped to dtype bounds." }, { "equation": "\\tau' = \\tau - b_s", "caption": "Threshold is shifted with the same per-scan background." } ], "diagram": "flowchart TD\n A[\"Receive per-scan background map\"] --> B[\"Skip background = 0\"]\n B --> C[\"Load latest edit\"]\n C --> D[\"Subtract background\"]\n D --> E[\"Save backfixed edit + lossy\"]\n E --> F[\"Update threshold and backfixed_shift\"]" }, "related": [ "homogenize-background" ], "video": null } ] }, { "id": "intensity", "title": "Intensity & thresholding", "methods": [ { "id": "apply-threshold", "title": "Align Thresholds", "keywords": [ "threshold", "align thresholds", "Otsu", "Li", "Kapur", "histogram matching", "percentile matching", "reference scan" ], "summary": "Align Thresholds computes and writes new segmentation thresholds into scan metadata without modifying voxel data. Supports copying a reference threshold, histogram-peak matching, percentile matching, and per-scan automatic methods (Otsu, Li, Kapur entropy). Can target all eligible voxel scans or only the selected scan, excluding preserved meshes, faulty scans, elastic registrations, and optionally the reference itself. Useful for harmonizing mesh extraction thresholds across a cohort before cleanup or registration.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "reference", "type": "string|null", "default": null, "description": "Reference scan name; required for same-threshold, histogram-matching, and percentile-matching methods." }, { "name": "method", "type": "string", "default": "same-threshold", "description": "Threshold strategy: same-threshold, histogram-matching, percentile-matching, otsus-method, li-method, or kapur-entropy." }, { "name": "onlyCurrentScan", "type": "boolean", "default": false, "description": "When true, update threshold only for selectedScan." }, { "name": "selectedScan", "type": "string|null", "default": null, "description": "Scan name to threshold when onlyCurrentScan is true." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter: off, exclude flagged scans, or only flagged scans." } ], "algorithm": { "text": "Load eligible voxel target scans and, for reference-based methods, Gaussian-smooth the reference volume and derive reference threshold, foreground volume, and variance. For each target scan, compute a threshold by method: copy reference value, shift reference threshold by histogram peak difference, match reference percentile rank, or run Otsu/Li/Kapur on smoothed scan data. Clamp below the scan minimum when needed, persist threshold to metadata JSON, and emit progress updates.", "math": [ { "equation": "T_{\\mathrm{scan}} = T_{\\mathrm{ref}}", "caption": "same-threshold: copy the reference threshold onto each target scan." }, { "equation": "T_{\\mathrm{scan}} = P_{\\mathrm{scan}}\\!\\left(\\mathrm{rank}(T_{\\mathrm{ref}}; I_{\\mathrm{ref}})\\right)", "caption": "percentile-matching: map the reference threshold’s percentile rank onto the target intensity distribution P_scan." }, { "equation": "T_{\\mathrm{scan}} = T_{\\mathrm{ref}} - (p_{\\mathrm{ref}} - p_{\\mathrm{scan}})", "caption": "histogram-matching: shift by the difference of intensity histogram peaks p_ref and p_scan." }, { "equation": "T^\\star = \\arg\\max_T \\left( H_{\\mathrm{bg}}(T) + H_{\\mathrm{fg}}(T) \\right)", "caption": "Kapur: choose T maximizing background + foreground Shannon entropy." } ], "diagram": "flowchart TD\n A[\"Select method\"] --> B{\"Needs reference?\"}\n B -->|\"yes\"| C[\"Load reference + threshold\"]\n B -->|\"no\"| D[\"Per-scan analysis only\"]\n C --> E[\"For each target scan\"]\n D --> E\n E --> F{\"method\"}\n F -->|\"same-threshold\"| G[\"Copy reference threshold\"]\n F -->|\"histogram-matching\"| H[\"Shift by peak difference\"]\n F -->|\"percentile-matching\"| I[\"Match percentile rank\"]\n F -->|\"otsus/li/kapur\"| J[\"Auto-threshold scan\"]\n G --> K[\"Write metadata threshold\"]\n H --> K\n I --> K\n J --> K" }, "related": [ "save-threshold-values", "match-histogram", "cleanup-mesh", "marchingcubes", "homogenize-background" ], "video": null, "images": [ { "caption": "A thresholded mask is overlaid on the source anatomy.", "path": "/static/docs-figures/docs_apply_threshold.gif" } ] }, { "id": "match-histogram", "title": "Intensity Normalization", "keywords": [ "intensity normalization", "histogram matching", "peak align", "tissue aware", "percentile normalize", "reference scan" ], "summary": "Normalizes voxel intensities across a project so subjects share a comparable intensity scale (shown in the Aurora UI as Intensity Normalization). Strategies include full foreground histogram matching to a reference, linear threshold alignment, tissue-aware soft-tissue and bone scaling, or standalone percentile clipping and rescaling. Saves new *_histmatched edits, updates thresholds in metadata, and skips preserved meshes, faulty scans, already histmatched scans, and elastic registrations. Useful before registration when scanners or reconstructions produce different intensity profiles.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "reference", "type": "string|null", "default": null, "description": "Reference scan name; required for reference-based methods." }, { "name": "method", "type": "string", "default": "histogram_match", "description": "Normalization method: histogram_match, peak_align, tissue_aware, or percentile_normalize." }, { "name": "percentile_low", "type": "number", "default": 2, "description": "Lower percentile used by percentile_normalize." }, { "name": "percentile_high", "type": "number", "default": 98, "description": "Upper percentile used by percentile_normalize." }, { "name": "onlyCurrentScan", "type": "boolean", "default": false, "description": "When true, normalize only selectedScan." }, { "name": "selectedScan", "type": "string|null", "default": null, "description": "Scan name to process when onlyCurrentScan is true." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter: off, exclude flagged scans, or only flagged scans." } ], "algorithm": { "text": "histogram_match: match foreground voxel intensities to the reference foreground using skimage.exposure.match_histograms. peak_align: add (T_ref - T_scan) to all voxels and clip to dtype range. tissue_aware: detect soft-tissue and bone peaks in reference and scan foreground, linearly rescale each range separately. percentile_normalize: clip the full volume to [p_low, p_high] and linearly map to the dtype range, remapping threshold through the same transform; reference scan is included. All methods save histmatched edits, lossy copies, updated thresholds, and copied masks/landmarks.", "math": [ { "equation": "v' = \\mathrm{clip}(v + T_{\\mathrm{ref}} - T_{\\mathrm{scan}})", "caption": "peak_align: translate intensities so the scan threshold/peak lines up with the reference." }, { "equation": "v' = \\frac{\\mathrm{clip}(v, p_L, p_H) - p_L}{p_H - p_L}\\,(M - m) + m", "caption": "percentile_normalize (recommended): stretch the [p_L, p_H] window onto dtype [m, M]; defaults p_L = 2nd percentile, p_H = 98th (UI-adjustable)." } ], "diagram": "flowchart TD\n A[\"Choose method\"] --> B{\"percentile_normalize?\"}\n B -->|\"yes\"| C[\"Clip + rescale each scan\"]\n B -->|\"no\"| D[\"Load reference scan\"]\n D --> E{\"method\"}\n E -->|\"histogram_match\"| F[\"Match foreground histograms\"]\n E -->|\"peak_align\"| G[\"Shift by threshold difference\"]\n E -->|\"tissue_aware\"| H[\"Rescale soft tissue and bone ranges\"]\n F --> I[\"Save histmatched edit\"]\n G --> I\n H --> I\n C --> I" }, "related": [ "apply-threshold", "n4-bias-correction", "homogenize-background" ], "video": null, "images": [ { "caption": "Intensities move from the backfixed state into the hist-matched distribution.", "path": "/static/docs-figures/docs_match_histogram.gif" } ] }, { "id": "n4-bias-correction", "title": "N4 Bias Correction", "keywords": [ "N4", "bias field", "intensity inhomogeneity", "ANTs", "MRI", "batch" ], "summary": "Corrects low-frequency MRI intensity inhomogeneity with ANTs N4 bias-field correction (Tustison et al., 2010) via ants.n4_bias_field_correction.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "filename", "type": "string|null", "default": null, "description": "Optional single scan name; when set, only that scan is corrected." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter for batch mode: off, exclude flagged scans, or only flagged scans." }, { "name": "parameters", "type": "object", "default": {}, "description": "Nested N4 tuning options passed to ANTs." }, { "name": "parameters.shrink_factor", "type": "integer", "default": 4, "description": "Downsampling factor for bias-field estimation speed." }, { "name": "parameters.convergence_iters", "type": "integer", "default": 50, "description": "Iterations per resolution level in the N4 convergence schedule." }, { "name": "parameters.convergence_tol_exp", "type": "integer", "default": 7, "description": "Exponent for convergence tolerance: tol = 10^(-convergence_tol_exp)." }, { "name": "parameters.spline_param", "type": "integer", "default": 200, "description": "B-spline control-point spacing parameter for the estimated bias field." } ], "algorithm": { "text": "1) Select eligible voxel scans; load the latest edit as float32. 2) Run N4 with defaults shrink_factor = 4, 50 iterations per level across 4 hierarchical levels, spline_distance = 200 mm, convergence tolerance = 1e-07 (overridable in the UI). 3) Clip the corrected volume to the original dtype range, save _n4 / bias-corrected edit plus lossy copy, propagate masks/landmarks, emit progress.", "math": [ { "equation": "I_{\\mathrm{corr}}(x) = I(x) / B(x)", "caption": "I: observed intensity; B: smooth bias field from iterative B-spline N4 fitting; x: voxel. Defaults: shrink 4, 50×4 iterations, spline distance 200 mm, tol 1e-07." } ], "diagram": "flowchart TD\n A[\"Select eligible scans\"] --> B[\"Load latest edit as float32\"]\n B --> C[\"Run ANTs N4 bias correction\"]\n C --> D[\"Clip to original dtype\"]\n D --> E[\"Save n4corrected edit + lossy\"]\n E --> F[\"Copy masks and landmarks\"]" }, "related": [ "match-histogram", "apply-threshold" ], "video": null, "images": [ { "caption": "Low-frequency MRI intensity shading flattens after N4 bias-field correction.", "path": "/static/docs-figures/docs_n4_bias_correction.gif" } ] }, { "id": "save-threshold-values", "title": "Save Threshold Values", "keywords": [ "threshold", "metadata", "manual", "per-scan", "UI values" ], "summary": "Persists user-edited threshold values from the UI into each scan's metadata JSON without modifying voxel data. Useful after manual threshold review in the viewer when values should be saved for mesh extraction, cleanup, or downstream preprocessing steps.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "values", "type": "object", "default": {}, "description": "Map of scan_name to objects containing a numeric threshold field to write into metadata." } ], "algorithm": { "text": "For each scan entry in values, load extracted/{scan_name}/{scan_name}.json, update the threshold field when present, and save the metadata file. Missing metadata files are reported as errors; successful updates are returned in updated_scans.", "math": [], "diagram": "" }, "related": [ "apply-threshold", "cleanup-mesh", "remove-background" ], "video": null } ] }, { "id": "metadata", "title": "Metadata Utilities", "methods": [ { "id": "get-rotation-matrix", "title": "Get Rotation Matrix", "keywords": [ "rotation", "orientation", "metadata", "rotated_by_matrix", "identity matrix" ], "summary": "Reads the 3×3 rotation matrix stored in scan metadata under rotated_by_matrix.rotation_matrix, or returns the identity matrix when metadata is missing or no rotation was recorded. Useful for frontend visualization, coordinate transforms, or applying/displaying prior orientation corrections without reloading the full volume.", "options": [ { "name": "directory", "type": "string", "default": null, "description": "Project root directory containing extracted scan folders." }, { "name": "scan_name", "type": "string", "default": null, "description": "Scan whose metadata JSON is queried for rotated_by_matrix.rotation_matrix." } ], "algorithm": { "text": "Open extracted/{scan_name}/{scan_name}.json. If rotated_by_matrix.rotation_matrix exists, return it. Otherwise return the 3×3 identity matrix. Missing metadata also yields identity with a success response.", "math": [], "diagram": "flowchart TD\n A[\"Read scan metadata JSON\"] --> B{\"rotation_matrix present?\"}\n B -->|\"yes\"| C[\"Return stored 3x3 matrix\"]\n B -->|\"no\"| D[\"Return identity matrix\"]" }, "related": [], "video": null } ] } ] }, { "id": "denoise-restore", "title": "Denoise & Restore", "subcategories": [ { "id": "batch-denoising", "title": "Batch Denoising", "methods": [ { "id": "denoise-all-scans", "title": "Denoise", "keywords": [ "denoise", "gaussian", "median", "non-local means", "nlm", "total variation", "wavelet", "butterworth", "batch", "nifti" ], "summary": "Applies a chosen denoising algorithm to every eligible voxel-based scan in a project directory, saving a new edit tagged _denoised with full and lossy NIfTI copies plus propagated masks and landmarks. Use this when scan data is noisy from acquisition or reconstruction and you want consistent filtering across subjects before segmentation, meshing, or visualization. Skips faulty scans, preserved PLY meshes, scans that already have a latest _denoised edit, and scans with elastic registration. Supports Gaussian blur, median filtering, Non-Local Means (2.5D CPU parallel, 3D CPU, or 3D GPU via nlm3d), Total Variation (Chambolle or Bregman), wavelet shrinkage, and Butterworth frequency filtering.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted/ scan folders." }, { "name": "algorithm", "type": "string", "default": "Gaussian", "description": "Denoising method: Gaussian, Median, NonLocalMeans, TotalVariationChambolle, TotalVariationBregman, Wavelet, or Butterworth." }, { "name": "parameters", "type": "object", "default": "{}", "description": "Algorithm-specific settings (see individual keys below)." }, { "name": "parameters.sigma", "type": "float", "default": "1.0 (Gaussian) / 0.1 (Wavelet)", "description": "Gaussian blur standard deviation, or wavelet noise estimate when algorithm is Wavelet." }, { "name": "parameters.truncate", "type": "float", "default": "4.0", "description": "Gaussian filter truncate radius (Gaussian only)." }, { "name": "parameters.h", "type": "float", "default": "0.1", "description": "NLM filter strength h on normalized [0,1] data (NonLocalMeans only)." }, { "name": "parameters.template_size", "type": "int", "default": "7", "description": "NLM patch (template) diameter in voxels (NonLocalMeans only)." }, { "name": "parameters.search_size", "type": "int", "default": "21", "description": "NLM search window radius in voxels (NonLocalMeans only)." }, { "name": "parameters.mode", "type": "string", "default": "2.5D", "description": "NLM execution mode: 2.5D (multi-axis slab average), 3D_CPU (skimage), or 3D_GPU (PyTorch nlm3d)." }, { "name": "parameters.weight", "type": "float", "default": "0.1", "description": "TV denoising weight (TotalVariationChambolle or TotalVariationBregman)." }, { "name": "parameters.eps", "type": "float", "default": "0.002", "description": "TV stopping tolerance (TotalVariationChambolle or TotalVariationBregman)." }, { "name": "parameters.max_num_iter", "type": "int", "default": "200", "description": "Maximum TV iterations (TotalVariationChambolle or TotalVariationBregman)." }, { "name": "parameters.wavelet", "type": "string", "default": "db1", "description": "Wavelet basis name (Wavelet only)." }, { "name": "parameters.level", "type": "int", "default": "1", "description": "Wavelet decomposition level (Wavelet only)." }, { "name": "parameters.cutoff", "type": "float", "default": "0.05", "description": "Butterworth cutoff frequency ratio (Butterworth only)." }, { "name": "parameters.order", "type": "int", "default": "2", "description": "Butterworth filter order (Butterworth only)." }, { "name": "parameters.high_pass", "type": "bool", "default": "true", "description": "If true, apply high-pass Butterworth; else low-pass (Butterworth only)." }, { "name": "parameters.squared_butterworth", "type": "bool", "default": "false", "description": "Use squared Butterworth response (Butterworth only)." }, { "name": "denoiseAutoMode", "type": "object", "default": "{}", "description": "Per-parameter auto-mode flags sent from the UI (currently logged but not applied server-side)." }, { "name": "onlyCurrentScan", "type": "bool", "default": "false", "description": "When true, restrict processing to selectedScan." }, { "name": "selectedScan", "type": "string|null", "default": "null", "description": "Scan folder name to process when onlyCurrentScan is true." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter mode (off / flagged-only / unflagged-only)." } ], "algorithm": { "text": "1) Enumerate extracted scan folders, excluding faulty metadata and preserved meshes. 2) Optionally filter to one scan, apply flag filter, and skip scans whose latest edit is already _denoised or that have elastic registration. 3) For each remaining scan, load the latest NIfTI edit (or original). 4) Normalize intensities to [0,1] when the chosen backend requires it, then rescale the result back to the original dtype range after denoising. 5) Apply the user-selected denoising method — Aurora dispatches to one of the backends below (library filters, or Aurora’s own 3D GPU Non-Local Means when mode is 3D_GPU). [[references]] 6) NonLocalMeans orchestration: with split=True, large volumes are tiled; mode 2.5D runs skimage NLM along three axes and averages; mode 3D_CPU runs skimage NLM on cubic chunks in parallel; mode 3D_GPU calls Aurora’s nlm3d CUDA implementation with an OOM tiling fallback. 7) Save {scan}_edit_{n}_denoised.nii.gz plus lossy copy, copy masks/landmarks, and broadcast WebSocket progress.", "math": [], "diagram": "flowchart TD\n A[\"POST directory + algorithm\"] --> B[\"Filter eligible scans\"]\n B --> C{\"onlyCurrentScan?\"}\n C -->|\"yes\"| D[\"Single scan\"]\n C -->|\"no\"| E[\"All scans\"]\n D --> F[\"Load latest NIfTI\"]\n E --> F\n F --> G{\"algorithm\"}\n G -->|\"Gaussian/Median/TV/Wavelet/Butterworth\"| H[\"skimage/scipy filter\"]\n G -->|\"NonLocalMeans\"| I{\"mode\"}\n I -->|\"2.5D\"| J[\"Parallel slab NLM + axis average\"]\n I -->|\"3D_CPU\"| K[\"skimage denoise_nl_means\"]\n I -->|\"3D_GPU\"| L[\"nlm3d CUDA + OOM fallback\"]\n H --> M[\"Save _denoised edit + lossy + masks\"]\n J --> M\n K --> M\n L --> M", "references": [ { "id": "gaussian", "label": "Gaussian", "blurb": "Isotropic Gaussian smoothing (SciPy).", "library": "SciPy", "symbol": "scipy.ndimage.gaussian_filter", "url": "https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.gaussian_filter.html" }, { "id": "median", "label": "Median", "blurb": "Rank-order median filter (scikit-image).", "library": "scikit-image", "symbol": "skimage.filters.median", "url": "https://scikit-image.org/docs/stable/api/skimage.filters.html#skimage.filters.median" }, { "id": "nonlocalmeans-cpu", "label": "NLM (2.5D / 3D CPU)", "blurb": "scikit-image Non-Local Means. Aurora tiles/parallelizes; the filter itself is the library call.", "library": "scikit-image", "symbol": "skimage.restoration.denoise_nl_means", "url": "https://scikit-image.org/docs/stable/api/skimage.restoration.html#skimage.restoration.denoise_nl_means" }, { "id": "nonlocalmeans-gpu", "label": "NLM (3D GPU)", "blurb": "Aurora’s own PyTorch/CUDA 3D Non-Local Means (nlm3d), not a third-party package API.", "library": "Aurora (in-house)", "symbol": "GPUTorchNLMDenoise.nlm3d", "entryId": "gpu-nlm3d" }, { "id": "tv-chambolle", "label": "TV Chambolle", "blurb": "Total-variation denoising (Chambolle).", "library": "scikit-image", "symbol": "skimage.restoration.denoise_tv_chambolle", "url": "https://scikit-image.org/docs/stable/api/skimage.restoration.html#skimage.restoration.denoise_tv_chambolle" }, { "id": "tv-bregman", "label": "TV Bregman", "blurb": "Total-variation denoising (split-Bregman).", "library": "scikit-image", "symbol": "skimage.restoration.denoise_tv_bregman", "url": "https://scikit-image.org/docs/stable/api/skimage.restoration.html#skimage.restoration.denoise_tv_bregman" }, { "id": "wavelet", "label": "Wavelet", "blurb": "BayesShrink soft-threshold wavelet denoising.", "library": "scikit-image", "symbol": "skimage.restoration.denoise_wavelet", "url": "https://scikit-image.org/docs/stable/api/skimage.restoration.html#skimage.restoration.denoise_wavelet" }, { "id": "butterworth", "label": "Butterworth", "blurb": "Frequency-domain Butterworth filter.", "library": "scikit-image", "symbol": "skimage.filters.butterworth", "url": "https://scikit-image.org/docs/stable/api/skimage.filters.html#skimage.filters.butterworth" } ] }, "related": [ "preview-denoise", "gpu-nlm3d", "restore-all-scans", "n4-bias-correction" ], "video": null, "images": [ { "caption": "Denoising suppresses local noise while retaining cranial anatomy.", "path": "/static/docs-figures/docs_denoise_all_scans.gif" } ] } ] }, { "id": "denoise-preview", "title": "Denoise Preview", "methods": [ { "id": "preview-denoise", "title": "Denoise preview", "keywords": [ "denoise", "preview", "patch", "slice", "interactive", "cache" ], "summary": "Interactive preview endpoint that denoises a 100×100×100 voxel patch centered at user-specified coordinates and returns normalized center Z-slice pixels for raw and denoised data. Use this to tune denoising parameters in the Aurora UI before running batch Denoise All Scans. Raw patches are cached (max one entry) by scan name and coordinates so parameter sweeps skip repeated disk I/O; denoising is recomputed on every request so slider changes appear immediately.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted/ scan folders." }, { "name": "scan_name", "type": "string", "default": "required", "description": "Scan folder name to preview." }, { "name": "algorithm", "type": "string", "default": "Gaussian", "description": "Same algorithm names as DenoiseAllScansView." }, { "name": "parameters", "type": "object", "default": "{}", "description": "Algorithm-specific settings for the preview patch." }, { "name": "parameters.sigma", "type": "float", "default": "1.0 (Gaussian) / 0.1 (Wavelet)", "description": "Gaussian sigma or wavelet noise estimate." }, { "name": "parameters.truncate", "type": "float", "default": "4.0", "description": "Gaussian truncate (Gaussian only)." }, { "name": "parameters.h", "type": "float", "default": "0.1", "description": "NLM filter strength (NonLocalMeans only)." }, { "name": "parameters.patch_size", "type": "int", "default": "7", "description": "NLM template patch size in voxels (NonLocalMeans only; maps to template_size internally)." }, { "name": "parameters.patch_distance", "type": "int", "default": "11", "description": "NLM search distance in voxels (NonLocalMeans only; maps to search_size internally)." }, { "name": "parameters.mode", "type": "string", "default": "2.5D", "description": "NLM mode passed to denoise_nl_means_module with split=False." }, { "name": "parameters.weight", "type": "float", "default": "0.1 (Chambolle) / 5.0 (Bregman)", "description": "TV denoising weight." }, { "name": "parameters.eps", "type": "float", "default": "0.0002 (Chambolle) / 0.001 (Bregman)", "description": "TV stopping tolerance." }, { "name": "parameters.max_num_iter", "type": "int", "default": "200 (Chambolle) / 100 (Bregman)", "description": "Maximum TV iterations." }, { "name": "parameters.wavelet", "type": "string", "default": "db1", "description": "Wavelet basis (Wavelet only)." }, { "name": "parameters.level", "type": "int", "default": "1", "description": "Wavelet level (Wavelet only)." }, { "name": "parameters.cutoff", "type": "float", "default": "0.005", "description": "Butterworth cutoff ratio (Butterworth only)." }, { "name": "parameters.order", "type": "int", "default": "2", "description": "Butterworth order (Butterworth only)." }, { "name": "parameters.high_pass", "type": "bool", "default": "false", "description": "High-pass vs low-pass Butterworth (Butterworth only)." }, { "name": "parameters.squared_butterworth", "type": "bool", "default": "true", "description": "Squared Butterworth response (Butterworth only)." }, { "name": "denoiseAutoMode", "type": "object", "default": "{}", "description": "Per-parameter auto-mode flags from the UI (received but not applied server-side)." }, { "name": "x", "type": "number", "default": "0", "description": "Patch center X coordinate in voxel space." }, { "name": "y", "type": "number", "default": "0", "description": "Patch center Y coordinate in voxel space." }, { "name": "z", "type": "number", "default": "0", "description": "Patch center Z coordinate in voxel space." } ], "algorithm": { "text": "1) Build cache key from scan_name and integer (x,y,z). 2) On cache miss, load latest edit or original NIfTI and extract a 100³ patch (zero-padded at boundaries). 3) On cache hit, reuse the cached raw patch. 4) Delegate to DenoiseAllScansView denoise helpers with preview-specific parameter names (patch_size/patch_distance for NLM). 5) Extract the center Z slice from raw and denoised volumes, min-max normalize to uint8, rotate 90° CCW for display orientation, flatten to lists, and return slice_shape plus from_cache flag.", "math": "", "diagram": "flowchart TD\n A[\"POST coords + algorithm\"] --> B{\"Cache hit?\"}\n B -->|\"no\"| C[\"Load NIfTI + extract 100³ patch\"]\n B -->|\"yes\"| D[\"Use cached raw patch\"]\n C --> E[\"Store raw patch in cache\"]\n D --> F[\"Apply denoise algorithm\"]\n E --> F\n F --> G[\"Center Z slice raw + denoised\"]\n G --> H[\"Normalize + rot90 + flatten\"]\n H --> I[\"Return center_slice arrays\"]" }, "related": [ "denoise-all-scans", "gpu-nlm3d" ], "video": null } ] }, { "id": "restoration", "title": "Restoration", "methods": [ { "id": "restore-all-scans", "title": "Restore", "keywords": [ "restore", "clahe", "contrast", "histogram equalization", "mask", "batch", "nifti" ], "summary": "Applies contrast restoration to eligible voxel scans and saves a new _restored edit with full and lossy NIfTI copies. Currently supports CLAHE (Contrast Limited Adaptive Histogram Equalization), which locally boosts contrast while limiting noise amplification — useful when scans look flat or low-contrast after thresholding or denoising. Optionally restricts CLAHE to voxels inside a segmentation mask (prefer_mask) so background stays unchanged. Skips faulty scans, preserved meshes, and scans whose latest edit is already _restored.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted/ scan folders." }, { "name": "algorithm", "type": "string", "default": "CLAHE", "description": "Restoration method (currently only CLAHE is implemented)." }, { "name": "parameters", "type": "object", "default": "{}", "description": "Algorithm-specific settings." }, { "name": "parameters.kernel_size", "type": "int", "default": "0", "description": "CLAHE local window size in voxels per axis. 0 means auto (skimage uses 1/8 of each dimension)." }, { "name": "parameters.clip_limit", "type": "float", "default": "0.01", "description": "CLAHE clip limit controlling contrast amplification (higher = stronger)." }, { "name": "parameters.nbins", "type": "int", "default": "256", "description": "Number of histogram bins for CLAHE." }, { "name": "onlyCurrentScan", "type": "bool", "default": "false", "description": "When true, restrict processing to selectedScan." }, { "name": "selectedScan", "type": "string|null", "default": "null", "description": "Scan folder name when onlyCurrentScan is true." }, { "name": "prefer_mask", "type": "bool", "default": "false", "description": "When true with CLAHE, apply restoration only inside the segmentation mask; outside voxels keep original intensities." }, { "name": "mask_label", "type": "int", "default": "1", "description": "Mask voxel value treated as foreground when prefer_mask is true." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Subject flag filter mode (off / flagged-only / unflagged-only)." } ], "algorithm": { "text": "1) Enumerate eligible voxel scans (same filtering pattern as denoise: faulty, meshes, flags, skip existing _restored). 2) Load latest NIfTI per scan and normalize to [0,1]. 3) Run skimage exposure.equalize_adapthist with kernel_size (auto when 0), clip_limit, and nbins. 4) Rescale restored values back to the original intensity range and dtype. 5) If prefer_mask, load the mask paired with the source edit and blend: restored inside mask_label voxels, original elsewhere. 6) Save {scan}_edit_{n}_restored.nii.gz, lossy copy, propagate masks and landmarks, broadcast progress.", "math": [ { "equation": "\\mathrm{clip\\_limit} = c \\cdot \\overline{h}", "caption": "CLAHE: c is the user clip_limit multiplier (default parameters.clip_limit = 0.01); h̄ is the mean histogram bin count in a tile; peaks above c·h̄ are redistributed before CDF mapping." } ], "diagram": "flowchart TD\n A[\"POST directory + CLAHE params\"] --> B[\"Filter eligible scans\"]\n B --> C[\"Load latest NIfTI\"]\n C --> D[\"Normalize to 0-1\"]\n D --> E[\"skimage equalize_adapthist\"]\n E --> F{\"prefer_mask?\"}\n F -->|\"yes\"| G[\"Blend with original outside mask_label\"]\n F -->|\"no\"| H[\"Use full restored volume\"]\n G --> I[\"Save _restored edit + lossy + masks\"]\n H --> I" }, "related": [ "denoise-all-scans" ], "video": null, "images": [ { "caption": "Local contrast restoration makes previously muted anatomy visible.", "path": "/static/docs-figures/docs_restore_all_scans.gif" } ] } ] }, { "id": "gpu-non-local-means", "title": "GPU Non-Local Means", "methods": [ { "id": "gpu-nlm3d", "title": "GPU 3D Non-Local Means (nlm3d)", "keywords": [ "nlm", "non-local means", "gpu", "cuda", "pytorch", "denoise", "3d", "tiling", "oom" ], "summary": "Aurora’s in-house PyTorch/CUDA 3D Non-Local Means (GPUTorchNLMDenoise.nlm3d → apply_nonlocal_means_3d_with_fallback), used when Denoise runs NonLocalMeans with mode 3D_GPU. Accepts a normalized [H,W,D] float tensor on GPU, compares patches in a search neighborhood with skimage-style noise compensation, and on CUDA OOM falls back to context-padded tiled processing. This is not a scikit-image call — the CPU/2.5D NLM paths use skimage.restoration.denoise_nl_means instead.", "options": [ { "name": "X", "type": "torch.Tensor [H,W,D]", "default": "required", "description": "Input 3D volume (typically float32, normalized 0–1, on CUDA)." }, { "name": "kernel_size", "type": "int", "default": "3", "description": "Search neighborhood diameter: number of offset voxels along each axis (odd integer; denoiseTools passes 2*search_size+1 for split GPU path or search_size directly in non-split path)." }, { "name": "std", "type": "float", "default": "1.0", "description": "Filter strength h (denoiseTools maps parameters.h here)." }, { "name": "kernel_size_mean", "type": "int", "default": "3", "description": "Mean-filter patch diameter used before distance computation (denoiseTools maps parameters.template_size here)." }, { "name": "sub_filter_size", "type": "int", "default": "256", "description": "Approximate number of neighbor offsets processed per streaming iteration in the memory-efficient path." }, { "name": "debug", "type": "bool", "default": "false", "description": "Enable verbose debug logging during NLM processing." } ], "algorithm": { "text": "nlm3d is an alias for apply_nonlocal_means_3d_with_fallback. Primary path: apply_nonlocal_means_3d_mem_efficient streams neighbor offsets through non_local_means_loop_index, comparing mean-filtered patches and weighting original unsmoothed neighbors by exp(-distance²/(2h²)) with skimage-style noise variance compensation. On RuntimeError containing 'out of memory', copies input to CPU, clears CUDA cache, picks tile size 96–160 based on free VRAM, and runs apply_nonlocal_means_3d_tiled: context-padded tiles are processed on GPU via _process_tile_streaming with per-offset streaming, then center results are stitched into the full volume.", "math": [ { "equation": "w(p,q) = \\exp\\!\\left(-\\frac{\\|P(p)-P(q)\\|^2}{2h^2}\\right)", "caption": "w: patch-similarity weight; P(·): local 3D patch; h: filter strength." }, { "equation": "\\hat{I}(p) = \\frac{\\sum_q w(p,q)\\, I(q)}{\\sum_q w(p,q)}", "caption": "Denoised value at p is the normalized weighted sum over search-window voxels q." } ], "diagram": "flowchart TD\n A[\"nlm3d tensor on CUDA\"] --> B[\"apply_nonlocal_means_3d_mem_efficient\"]\n B --> C{\"OOM?\"}\n C -->|\"no\"| D[\"Return denoised volume\"]\n C -->|\"yes\"| E[\"Move to CPU + empty_cache\"]\n E --> F[\"Pick tile size from free VRAM\"]\n F --> G[\"apply_nonlocal_means_3d_tiled\"]\n G --> H[\"Stream offsets per tile\"]\n H --> D" }, "related": [ "denoise-all-scans", "preview-denoise" ], "video": null, "images": [ { "caption": "Zoomed detail shows non-local-means removing grain while edges stay sharp.", "path": "/static/docs-figures/docs_gpu_nlm3d.gif" } ] } ] } ] }, { "id": "masks-labels", "title": "Masks & Labels", "subcategories": [ { "id": "mask-io", "title": "Mask I/O", "methods": [ { "id": "save-mask", "title": "Save Mask", "keywords": [ "mask", "save", "brush", "nifti", "full resolution", "lossy", "binary" ], "summary": "Persists a painted or edited voxel mask from the Aurora canvas to disk as a NIfTI sidecar ({edit}.nii.mask.gz). Accepts either a compact binary upload or a legacy JSON uint8 array. Saves at the resolution you are currently editing (full or lossy) and automatically generates a complementary-resolution mask by nearest-neighbor zoom so both preview and export resolutions stay in sync.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory for the project or subject collection." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas masks." }, { "name": "edit", "type": "str", "default": "null", "description": "Edit stem without .nii.gz (e.g. scan_edit_1). When omitted, defaults to the raw scan / scan_lossy pair." }, { "name": "fullResolutionState", "type": "bool | str", "default": "false", "description": "True when the canvas is editing the full-resolution volume; false for the lossy preview volume. Multipart string values true/1/yes/on are accepted." }, { "name": "maskData", "type": "file | list[int]", "default": "required", "description": "Flat uint8 voxel array in frontend order (depth, height, width with Y/Z flipped). Prefer multipart file upload; JSON integer list is supported for legacy clients." } ], "algorithm": { "text": "1) Resolve source/target edit names from fullResolutionState and edit stem. 2) Load scan metadata to read resolution_factor from lossy_compression. 3) Reshape flat mask bytes to NIfTI (width, height, depth) with frontend axis flips. 4) Save source-resolution mask as {source_edit}.nii.mask.gz. 5) If the complementary image exists, scale the mask with scipy.ndimage.zoom (order=0) to that shape and save the paired mask.", "math": [ { "equation": "z_i = t_i / s_i", "caption": "z_i: per-axis zoom factor; t_i: target shape; s_i: source mask shape; nearest-neighbor zoom (order=0) follows." } ], "diagram": "flowchart TD\n A[\"Flat maskData\"] --> B{\"fullResolutionState?\"}\n B -->|\"true\"| C[\"Save full-res mask\"]\n B -->|\"false\"| D[\"Save lossy mask\"]\n C --> E[\"Zoom to lossy shape\"]\n D --> F[\"Zoom to full shape\"]\n E --> G[\"Write .nii.mask.gz pair\"]\n F --> G" }, "related": [ "load-mask", "upload-mask", "apply-mask-to-image" ], "video": null }, { "id": "load-mask", "title": "Load Mask", "keywords": [ "mask", "load", "brush", "binary", "canvas" ], "summary": "Loads a saved mask from disk back into the Aurora canvas. Returns raw uint8 bytes (application/octet-stream) instead of a JSON array for performance on large volumes. Response headers carry the mask filename and NIfTI dimensions so the frontend can reconstruct the flat brush buffer.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas masks." }, { "name": "edit", "type": "str", "default": "null", "description": "Edit stem without .nii.gz. When omitted, uses filename (full) or filename_lossy (lossy)." }, { "name": "fullResolutionState", "type": "bool", "default": "false", "description": "True to load the full-resolution mask; false for the lossy mask." } ], "algorithm": { "text": "1) Derive mask_edit from edit and fullResolutionState (insert or strip _lossy as needed). 2) Temporarily rename .nii.mask.gz to .nii.gz for nibabel. 3) Load NIfTI, reverse frontend axis flips (Y/Z reversed, depth-major flatten), round to uint8. 4) Return raw bytes with X-Mask-Name and X-Mask-Dimensions headers. 5) Restore the custom extension.", "math": [], "diagram": "flowchart TD\n A[\"Request edit + resolution\"] --> B[\"Resolve .nii.mask.gz path\"]\n B --> C[\"Temp rename for nibabel\"]\n C --> D[\"Load + axis invert\"]\n D --> E[\"HttpResponse octet-stream\"]\n E --> F[\"Restore extension\"]" }, "related": [ "save-mask", "download-mask" ], "video": null }, { "id": "upload-mask", "title": "Upload Mask", "keywords": [ "mask", "upload", "nifti", "import", "external" ], "summary": "Imports an external NIfTI mask file (.nii or .nii.gz) into a subject folder. Always stores the uploaded volume as the full-resolution mask and automatically downscales a complementary lossy mask when the lossy image exists, keeping both resolutions aligned for canvas editing.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas masks." }, { "name": "edit", "type": "str", "default": "required", "description": "Lossy edit name (e.g. scan_lossy.nii.gz). Defaults to {filename}_lossy.nii.gz when empty or null." }, { "name": "maskFile", "type": "file", "default": "required", "description": "Uploaded NIfTI mask; must end with .nii or .nii.gz." } ], "algorithm": { "text": "1) Validate extension and derive full_edit / lossy_edit from edit. 2) Read resolution_factor from subject metadata. 3) Save uploaded NIfTI as {full_edit}.nii.mask.gz. 4) If {lossy_edit}.nii.gz exists, nearest-neighbor zoom the mask to lossy dimensions and save {lossy_edit}.nii.mask.gz.", "math": [ { "equation": "M_{\\mathrm{lossy}} = \\mathrm{zoom}(M,\\, [L_i/F_i],\\, order{=}0)", "caption": "M: full-resolution mask; L_i/F_i: lossy vs full shape ratio per axis; order=0: nearest-neighbor label preservation." } ], "diagram": "flowchart TD\n A[\"Uploaded .nii.gz\"] --> B[\"Save full-res mask\"]\n B --> C{\"Lossy image exists?\"}\n C -->|\"yes\"| D[\"Zoom mask to lossy shape\"]\n C -->|\"no\"| E[\"Done\"]\n D --> E" }, "related": [ "save-mask", "download-mask", "delete-mask" ], "video": null }, { "id": "download-mask", "title": "Download Mask", "keywords": [ "mask", "download", "export", "nifti", "full resolution" ], "summary": "Downloads the full-resolution mask file for a given edit as an attachment. Strips _lossy from the edit name so users always receive the native full-res sidecar regardless of which resolution they were viewing.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas masks." }, { "name": "edit", "type": "str", "default": "required", "description": "Edit name (may include .nii.gz and/or _lossy). Defaults to {filename}.nii.gz when empty or null." } ], "algorithm": { "text": "1) Strip .nii.gz and _lossy to get full_edit. 2) Locate {full_edit}.nii.mask.gz under extracted/{filename} or atlas/. 3) Temporarily rename to standard .nii.gz extension. 4) Stream file via FileResponse with Content-Disposition attachment header. 5) Restore custom extension.", "math": null, "diagram": "flowchart TD\n A[\"edit param\"] --> B[\"Derive full_edit\"]\n B --> C[\"Locate .nii.mask.gz\"]\n C --> D[\"Temp rename + FileResponse\"]\n D --> E[\"Restore extension\"]" }, "related": [ "load-mask", "upload-mask" ], "video": null }, { "id": "delete-mask", "title": "Delete Mask", "keywords": [ "mask", "delete", "remove", "full resolution", "lossy" ], "summary": "Removes both full-resolution and lossy mask sidecars for a given edit. Use this when clearing brush data before re-labeling or when discarding an obsolete mask without deleting the underlying image edits.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas masks." }, { "name": "edit", "type": "str", "default": "required", "description": "Edit name (may include .nii.gz and/or _lossy). Defaults to {filename}.nii.gz when empty or null." } ], "algorithm": { "text": "1) Derive full_edit and lossy_edit from edit. 2) Build paths for {full_edit}.nii.mask.gz and {lossy_edit}.nii.mask.gz. 3) Delete each file that exists. 4) Return 404 if neither file was found.", "math": null, "diagram": "flowchart TD\n A[\"edit param\"] --> B[\"full + lossy mask paths\"]\n B --> C[\"Delete existing files\"]\n C --> D{\"Any deleted?\"}\n D -->|\"yes\"| E[\"200 OK\"]\n D -->|\"no\"| F[\"404\"]" }, "related": [ "save-mask", "copy-mask-to-reference" ], "video": null }, { "id": "copy-mask-to-reference", "title": "Copy Mask to Reference", "keywords": [ "mask", "copy", "reference", "transfer", "batch alignment" ], "summary": "Copies a saved mask from one subject scan to another (reference) scan in the same project. Copies both full-resolution and lossy sidecars when present, overwriting destination files atomically via a temp file. Use after rigid alignment when a mask drawn on one subject should seed labeling on a homologous reference.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory containing extracted/ subjects." }, { "name": "selected_scan", "type": "str", "default": "required", "description": "Source subject folder name." }, { "name": "selected_edit_mask", "type": "str", "default": "required", "description": "Source mask filename (typically the lossy mask name passed from the frontend)." }, { "name": "reference_scan", "type": "str", "default": "required", "description": "Destination subject folder name; must differ from selected_scan." }, { "name": "reference_edit_mask", "type": "str", "default": "required", "description": "Destination mask filename stem (lossy form from frontend)." } ], "algorithm": { "text": "1) Reject if selected_scan equals reference_scan. 2) Locate full and lossy source masks under extracted/{selected_scan}/. 3) For each found mask, copy to extracted/{reference_scan}/ with the reference edit name (strip _lossy for full, keep for lossy). 4) Use shutil.copy2 to a .tmp file, verify size, then atomic os.replace. 5) Report copied and overwritten filenames.", "math": null, "diagram": "flowchart TD\n A[\"Source scan + mask\"] --> B[\"Find full + lossy masks\"]\n B --> C[\"Map to reference edit names\"]\n C --> D[\"Atomic copy via .tmp\"]\n D --> E[\"Return copied/overwritten list\"]" }, "related": [ "save-mask", "apply-mask-to-image" ], "video": null }, { "id": "list-mask-folders", "title": "List Mask Folders", "keywords": [ "mask", "folders", "list", "batch", "organization" ], "summary": "Returns the names of existing mask output folders under {directory}/masks/. Use this to populate UI dropdowns before applying or batch-applying masks, or to confirm whether a named mask collection already exists.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory whose masks/ subfolder will be listed." } ], "algorithm": { "text": "1) Build path {directory}/masks/. 2) If the directory does not exist, return an empty list with count 0. 3) Otherwise list subdirectory names, sort alphabetically, and return mask_folders and count.", "math": null, "diagram": "flowchart TD\n A[\"directory\"] --> B[\"masks/ exists?\"]\n B -->|\"no\"| C[\"Empty list\"]\n B -->|\"yes\"| D[\"List + sort subdirs\"]\n D --> E[\"Return mask_folders\"]" }, "related": [ "apply-mask-to-image", "batch-apply-mask-to-scans" ], "video": null } ] }, { "id": "apply-mask", "title": "Apply Mask", "methods": [ { "id": "apply-mask-to-image", "title": "Apply masks (single)", "keywords": [ "mask", "apply", "zero", "crop", "mask folder", "edit_0" ], "summary": "Applies a saved full-resolution mask to its corresponding image edit and writes a new masked dataset into masks/{mask_folder}/. Non-masked voxels are zeroed, the result is saved as edit_0 (preserving any edit suffix), a lossy preview is generated, metadata is copied without lossy_compression, and elastic transforms from the next edit are renumbered to edit_1 when present.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas data." }, { "name": "edit", "type": "str", "default": "required", "description": "Source edit being masked (e.g. scan_lossy.nii.gz). Defaults to {filename}_lossy.nii.gz when empty or null." }, { "name": "mask_folder", "type": "str", "default": "required", "description": "Target folder name under masks/ where the masked copy will be stored." }, { "name": "create_new", "type": "bool", "default": "false", "description": "When true, logs initialization of a new mask folder (directory structure is always created as needed)." } ], "algorithm": { "text": "1) Create masks/{mask_folder}/extracted/{filename}/ (or atlas/). 2) Copy project_settings.json once. 3) If the source edit has a following elastic edit, copy and rename elastic fwd/inv fields to edit_1. 4) Write an empty top-level {filename}.nii.gz placeholder. 5) Copy metadata minus lossy_compression. 6) Load full-res mask and image; zero voxels where mask == 0. 7) Save {filename}_edit_0{suffix}.nii.gz plus empty {filename}.nii.gz and {filename}_lossy.nii.gz placeholders. 8) Generate lossy edit_0 via RegistrationTools.save_as_lossy_nifti (fallback: scipy zoom).", "math": [ { "equation": "I' = I \\cdot \\mathbf{1}[M > 0]", "caption": "I: image; M: mask; voxels with M≤0 are zeroed." } ], "diagram": "flowchart TD\n A[\"Load mask + image\"] --> B[\"binary_mask = mask > 0\"]\n B --> C[\"Zero outside mask\"]\n C --> D[\"Save edit_0 full + lossy\"]\n D --> E[\"Copy metadata + elastic to mask folder\"]" }, "related": [ "batch-apply-mask-to-scans", "list-mask-folders", "save-mask", "load-mask" ], "video": null }, { "id": "batch-apply-mask-to-scans", "title": "Apply masks", "keywords": [ "mask", "batch", "label", "apply", "dilation", "flag filter", "progress" ], "summary": "Applies a single label value from each subject's latest full-resolution mask across all eligible scans in extracted/ (and atlas when present). Subjects without the label or without a mask are skipped. Optionally dilates the label region before zeroing everything outside it. Writes results into masks/{mask_folder}/ and broadcasts WebSocket progress updates.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory containing extracted/." }, { "name": "label", "type": "int", "default": "required", "description": "Mask label value (0–255) to isolate and apply." }, { "name": "mask_folder", "type": "str", "default": "required", "description": "Target folder name under masks/." }, { "name": "create_new", "type": "bool", "default": "false", "description": "When true, explicitly creates the masks/{mask_folder}/ directory before processing." }, { "name": "dilation", "type": "int", "default": "0", "description": "Morphological dilation of the label mask in voxels before applying (clamped 0–100)." }, { "name": "flagFilter", "type": "str", "default": "off", "description": "Batch subject filter for flagged scans: \"off\" (all), \"exclude\" (skip flagged), or \"only\" (flagged only)." } ], "algorithm": { "text": "1) Enumerate voxel subjects in extracted/ (+ atlas), excluding faulty scans, preserved meshes, and applying flagFilter. 2) For each subject, find the latest full-res .nii.mask.gz (no _lossy). 3) Skip if no mask or label not in unique values. 4) Build label_mask = (mask == label), optionally dilate. 5) Multiply image by label_mask; save as edit_0 under masks/{mask_folder}/extracted/{subject}/. 6) Copy metadata (no lossy_compression), create lossy edit_0, copy/rename elastic transforms and masked elastic copies when present. 7) Emit progress_group WebSocket events.", "math": [ { "equation": "I' = I \\cdot \\mathrm{dilate}(M = \\ell,\\, d)", "caption": "ℓ: selected label; d: dilation radius in voxels; dilate expands the label before masking." } ], "diagram": "flowchart TD\n A[\"All subjects\"] --> B[\"Filter flags/meshes/faulty\"]\n B --> C[\"Latest full-res mask\"]\n C --> D{\"Label present?\"}\n D -->|\"no\"| E[\"Skip\"]\n D -->|\"yes\"| F[\"Optional dilation\"]\n F --> G[\"Multiply + save edit_0\"]\n G --> H[\"Next subject\"]" }, "related": [ "apply-mask-to-image", "batch-mask-out-scans", "list-mask-folders" ], "video": null } ] }, { "id": "mask-out", "title": "Mask Out", "methods": [ { "id": "mask-out-labels", "title": "Mask Out Labels", "keywords": [ "mask out", "delete", "isolate", "labels", "edit", "landmarks", "invert" ], "summary": "Creates a new in-place edit by zeroing voxels in the current image that match one or more mask labels (or, with invert=true, keeping only those voxels). Saves {filename}_edit_{N}_masked.nii.gz plus a lossy companion, copies and downsamples the mask to the new edit, and copies landmarks unchanged because geometry is not transformed.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject or scan folder name; use \"atlas\" for atlas data." }, { "name": "edit", "type": "str", "default": "filename", "description": "Current edit name (e.g. scan_edit_2_elastic.nii.gz). Defaults to filename when omitted." }, { "name": "labels", "type": "list[int]", "default": "required", "description": "Non-empty array of positive label integers to mask out (e.g. [1] or [1, 2, 3])." }, { "name": "fullResolutionState", "type": "bool", "default": "false", "description": "Resolution context from the frontend (mask and image are always loaded at full resolution regardless)." }, { "name": "invert", "type": "bool", "default": "false", "description": "False: zero labeled voxels (mask out). True: zero everything except labeled voxels (isolate)." } ], "algorithm": { "text": "1) Load full-res mask and image for edit. 2) Build combined_mask = OR(mask == label) for all labels; error if zero voxels. 3) If invert: zero ~combined_mask; else zero combined_mask. 4) Find next free edit number N. 5) Save {filename}_edit_{N}_masked.nii.gz. 6) Create lossy version via RegistrationTools.save_as_lossy_nifti. 7) Copy source mask to new edit; downsample lossy mask by resolution_factor stride slices. 8) Copy landmarks and landmark_distances JSON without transformation.", "math": [ { "equation": "C = \\bigvee_i (M = \\ell_i)", "caption": "C: combined label mask; ℓ_i: selected labels; ∨: OR over label indicators." }, { "equation": "I' = I \\cdot (C\\ \\mathrm{if\\ invert\\ else}\\ \\neg C)", "caption": "invert: keep the labeled region; otherwise zero it out." } ], "diagram": "flowchart TD\n A[\"labels + edit\"] --> B[\"Load full mask + image\"]\n B --> C[\"combined_mask OR labels\"]\n C --> D{\"invert?\"}\n D -->|\"false\"| E[\"Zero labeled voxels\"]\n D -->|\"true\"| F[\"Zero unlabeled voxels\"]\n E --> G[\"Save edit_N_masked + lossy\"]\n F --> G\n G --> H[\"Copy mask + landmarks\"]" }, "related": [ "batch-mask-out-scans", "save-mask", "get-label-names" ], "video": null }, { "id": "batch-mask-out-scans", "title": "Batch Mask Out Scans", "keywords": [ "mask out", "batch", "label", "elastic", "overwrite", "dilation", "flag filter" ], "summary": "Runs mask-out on every eligible subject in extracted/ (and atlas) for a single label, creating new _masked edits in-place rather than in a mask subfolder. Handles elastic-edit insertion by bumping elastic files forward, supports optional dilation, and can overwrite an existing masked-before-elastic slot when overwrite=true. Skips mesh-only and flagged subjects per flagFilter.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory containing extracted/." }, { "name": "label", "type": "int", "default": "required", "description": "Mask label value (0–255) to mask out or isolate." }, { "name": "invert", "type": "bool", "default": "false", "description": "False: zero labeled voxels. True: keep only labeled voxels." }, { "name": "overwrite", "type": "bool", "default": "false", "description": "When true, replace an existing masked edit inserted before elastic using the prior edit as source; when false, skip subjects where that insertion already exists." }, { "name": "dilation", "type": "int", "default": "0", "description": "Morphological dilation of the label mask in voxels before applying (clamped 0–100)." }, { "name": "flagFilter", "type": "str", "default": "off", "description": "Batch subject filter for flagged scans: \"off\", \"exclude\", or \"only\"." } ], "algorithm": { "text": "1) Enumerate and filter subjects (faulty, mesh-only, flagFilter). 2) For each subject, detect next edit slot and whether latest edit is elastic or already masked. 3) Skip if masked insertion exists and overwrite=false. 4) Select source mask (latest, or edit before elastic/overwrite slot). 5) combined_mask = (mask == label), optional dilation; apply invert logic to image. 6) If elastic follows: bump elastic files E→E+1, insert masked at E, mask elastic copies in-place. 7) Save edit_N_masked + lossy, copy/downsample mask, copy landmarks. 8) Fix lossy_compression metadata after elastic bump. 9) Broadcast progress_group events.", "math": [ { "equation": "C = \\mathrm{dilate}(M = \\ell,\\, d)", "caption": "C: dilated label region; ℓ: label; d: dilation." }, { "equation": "I'(v) = 0\\ \\mathrm{where}\\ (C\\ \\mathrm{xor\\ invert})(v)", "caption": "Zero voxels inside C (or outside C when invert is set)." } ], "diagram": "flowchart TD\n A[\"Each subject\"] --> B{\"Elastic is latest?\"}\n B -->|\"yes\"| C[\"Bump elastic + insert masked before\"]\n B -->|\"no\"| D[\"Use next edit slot\"]\n C --> E[\"Apply label mask to image\"]\n D --> E\n E --> F[\"Save _masked edit + lossy\"]\n F --> G[\"Copy mask + landmarks\"]" }, "related": [ "mask-out-labels", "batch-apply-mask-to-scans" ], "video": null } ] } ] }, { "id": "segmentation-edges", "title": "Segmentation & Edges", "subcategories": [ { "id": "ai-segmentation-section", "title": "AI Segmentation", "methods": [ { "id": "get-ai-models", "title": "Get AI Models", "keywords": [ "segmentation", "ai", "medsam2", "models", "list" ], "summary": "Returns the list of AI segmentation models available in the Aurora pipeline. Call this before RunAISegmentationView so the UI can populate a model picker. Currently returns a hardcoded list containing MedSAM2.", "options": [], "algorithm": { "text": "GET handler with no request parameters. Returns a JSON object with models (string array) and count (integer). On failure returns HTTP 500 with an error message.", "math": "", "diagram": "flowchart TD\n A[\"GET / GetAIModelsView\"] --> B[\"Build available_models list\"]\n B --> C[\"Return models + count\"]" }, "related": [ "run-ai-segmentation", "ai-segmentation", "medsam2-segmentation" ], "video": null }, { "id": "run-ai-segmentation", "title": "Run AI Segmentation", "keywords": [ "segmentation", "ai", "medsam2", "mask", "label", "nifti" ], "summary": "Runs AI-assisted segmentation on a subject scan and merges the result into the existing mask at a chosen label. Loads the full-resolution NIfTI image and mask from the extracted subject folder (or atlas), dispatches to ai_segmentation, then saves both full- and lossy-resolution masks. Use when a user has seeded a rough label region and wants MedSAM2 to refine or expand it.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory containing extracted/ or atlas/ folders." }, { "name": "subject", "type": "str", "default": "required", "description": "Subject folder name or 'atlas' for atlas data." }, { "name": "edit", "type": "str | null", "default": "null", "description": "Edit filename (e.g. scan_lossy.nii.gz). When null, uses the raw subject image as base." }, { "name": "model", "type": "str", "default": "required", "description": "AI model name to run (currently only 'MedSAM2' is supported)." }, { "name": "current_label", "type": "int", "default": "required", "description": "Label number (1–255) to segment and write into the mask. 0 is reserved for background." } ], "algorithm": { "text": "1) Validate required POST fields and coerce current_label to int in range 1–255. 2) Resolve base_path (extracted/{subject} or atlas/). 3) Load {full_edit}.nii.gz image and existing or empty {full_edit}.nii.mask.gz mask (via temporary _mask.nii.gz rename for nibabel). 4) Call ai_segmentation(image, mask, current_label, model). 5) Save merged mask at full resolution, then downscale to lossy resolution with scipy.ndimage.zoom (order=0). 6) Return message, segmentation_label, voxels_segmented, and model_used.", "math": [], "diagram": "flowchart TD\n A[\"POST RunAISegmentationView\"] --> B[\"Validate params\"]\n B --> C[\"Load image + mask NIfTI\"]\n C --> D[\"ai_segmentation\"]\n D --> E[\"Merge at current_label\"]\n E --> F[\"Save full mask\"]\n F --> G[\"Scale + save lossy mask\"]\n G --> H[\"Return stats\"]" }, "related": [ "get-ai-models", "ai-segmentation", "medsam2-segmentation", "save-mask" ], "video": null }, { "id": "ai-segmentation", "title": "AI Segmentation Dispatcher", "keywords": [ "segmentation", "ai", "medsam2", "dispatch", "mask" ], "summary": "Internal dispatcher that routes AI segmentation requests to the correct model implementation. Currently supports MedSAM2 only; raises an exception for unsupported model names. Called by RunAISegmentationView after image and mask volumes are loaded.", "options": [ { "name": "image_data_3d", "type": "np.ndarray", "default": "required", "description": "3D medical image volume. Shape (width, height, depth) or (D, H, W); typically uint8 or float32." }, { "name": "mask_data_3d", "type": "np.ndarray", "default": "required", "description": "3D label mask volume, same shape as image. dtype uint8; 0 = background." }, { "name": "target_label", "type": "int", "default": "required", "description": "Label number to segment (1–255)." }, { "name": "model", "type": "str", "default": "required", "description": "Model name. Supported: 'MedSAM2'." } ], "algorithm": { "text": "If model == 'MedSAM2', delegate to medsam2_segmentation. Otherwise raise Exception with 'Model {model} not supported'. Returns the merged 3D mask array.", "math": "", "diagram": "flowchart TD\n A[\"ai_segmentation\"] --> B{\"model == MedSAM2?\"}\n B -->|\"yes\"| C[\"medsam2_segmentation\"]\n B -->|\"no\"| D[\"Raise unsupported model error\"]\n C --> E[\"Return result_mask\"]" }, "related": [ "run-ai-segmentation", "medsam2-segmentation", "get-ai-models" ], "video": null, "images": [ { "caption": "MedSAM2 brain-label overlay on the matching cleaned embryo edit slice.", "path": "/static/docs-figures/docs_ai_segmentation.gif" } ] }, { "id": "medsam2-segmentation", "title": "MedSAM2 Segmentation", "keywords": [ "medsam2", "segmentation", "bounding box", "ensemble", "mask" ], "summary": "Runs MedSAM2 on a 3D volume using the existing mask voxels at target_label to define a bounding box and binary reference mask. Segments within that region, keeps the largest connected component, uses a 3-view ensemble, then merges the result back into the mask at target_label only. Returns the original mask unchanged if no voxels carry the target label.", "options": [ { "name": "image_data_3d", "type": "np.ndarray", "default": "required", "description": "3D image volume, shape (D, H, W)." }, { "name": "mask_data_3d", "type": "np.ndarray", "default": "required", "description": "3D mask volume, shape (D, H, W), dtype uint8." }, { "name": "target_label", "type": "int", "default": "required", "description": "Label whose voxels define the bounding box and reference region." } ], "algorithm": { "text": "1) Find all voxels where mask == target_label; compute axis-aligned bounding box (d_min..d_max, h_min..h_max, w_min..w_max) in (z,y,x) order. 2) Build binary reference_mask (1 at target_label, 0 elsewhere). 3) Instantiate MedSAM2Segmenter(force_cpu=False). 4) Call segmenter.segment with bbox_min/bbox_max, reference mask, use_largest_cc=True, existing_mask_as_reference=True, ensemble=True. 5) Copy mask and set result_mask[segmentation_mask==1] = target_label.", "math": [ { "equation": "L = \\{(d,h,w) \\mid M_{dhw} = \\ell\\}", "caption": "L: voxels of the prompt label ℓ in mask M." }, { "equation": "b_{\\min} = \\min L", "caption": "Lower corner of the axis-aligned bounding box over label voxels L." }, { "equation": "b_{\\max} = \\max L", "caption": "Upper corner of the box used as the MedSAM2 spatial prompt." } ], "diagram": "flowchart TD\n A[\"Find target_label voxels\"] --> B{\"Bbox empty?\"}\n B -->|\"yes\"| C[\"Return mask copy\"]\n B -->|\"no\"| D[\"Build reference binary mask\"]\n D --> E[\"MedSAM2Segmenter.segment\"]\n E --> F[\"Merge into result_mask at target_label\"]\n F --> G[\"Return result_mask\"]" }, "related": [ "ai-segmentation", "run-ai-segmentation" ], "video": null, "images": [ { "caption": "Paint three mid-plane intensity seeds, lift them into 3D, propagate per axis (cyan / magenta / amber), then read the vote map — grey keeps ≥2 votes, red drops the rest — before orbiting the final surface.", "path": "/static/docs-figures/docs_medsam2_segmentation.gif" } ] } ] }, { "id": "edge-detection-section", "title": "Edge Detection", "methods": [ { "id": "edge-detection-slice-preview", "title": "Edge Detection Slice Preview", "keywords": [ "edge", "hysteresis", "preview", "patch", "morphology" ], "summary": "Fast 2D slice preview of hysteresis edge detection on a 100³ patch centered at (x,y,z). Uses the same global volume min/max normalization as the full mesh pipeline so slider tuning matches the final result. Caches raw patch and v_min/v_max per scan/edit/position/resolution (max 3 entries).", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject filename or 'atlas'." }, { "name": "edit", "type": "str", "default": "optional", "description": "Edit NIfTI filename; used to resolve volume path." }, { "name": "full_resolution", "type": "bool", "default": "false", "description": "Load full-resolution volume instead of lossy." }, { "name": "x", "type": "float", "default": "0", "description": "Patch center X coordinate in volume indices." }, { "name": "y", "type": "float", "default": "0", "description": "Patch center Y coordinate in volume indices." }, { "name": "z", "type": "float", "default": "0", "description": "Patch center Z coordinate in volume indices." }, { "name": "low", "type": "float", "default": "0.10", "description": "Hysteresis low threshold on normalized [0,1] intensities." }, { "name": "high", "type": "float", "default": "0.30", "description": "Hysteresis high threshold on normalized [0,1] intensities." }, { "name": "closing_radius", "type": "int", "default": "3", "description": "Radius (voxels) of the spherical structuring element for binary closing." } ], "algorithm": { "text": "1) Load or cache 100³ patch and global v_min/v_max. 2) Normalize patch: (patch - v_min)/(v_max - v_min). 3) apply_hysteresis_threshold(low, high). 4) binary_closing with morphology.ball(closing_radius). 5) Return flat uint8 center-Z slices for raw, edges, closed, and filled stages plus patch/volume intensity stats.", "math": [ { "equation": "\\hat{v} = (v - v_{\\min}) / (v_{\\max} - v_{\\min})", "caption": "Per-slice min–max normalization before the edge detector." }, { "equation": "\\mathrm{keep}\\ \\hat{v} \\ge t_{\\mathrm{high}}\\ \\mathrm{or\\ connected\\ through}\\ \\hat{v} \\ge t_{\\mathrm{low}}", "caption": "Hysteresis: strong edges (≥ t_high) plus weaker edges (≥ t_low) connected to them." } ], "diagram": "flowchart TD\n A[\"POST slice preview\"] --> B[\"Load/cache 100³ patch\"]\n B --> C[\"Global min-max normalize\"]\n C --> D[\"Hysteresis threshold\"]\n D --> E[\"Binary closing ball r\"]\n E --> F[\"Return center-Z slices flat\"]" }, "related": [ "edge-detection" ], "video": null }, { "id": "edge-detection", "title": "Edge Detection", "keywords": [ "edge", "hysteresis", "marching cubes", "mesh", "mask", "morphology" ], "summary": "Full-volume edge detection pipeline: hysteresis thresholding, morphological closing, optional largest-component filtering, then either preview GLB mesh via marching cubes or save the result as a segmentation mask label. When saving permanently from a lossy preview, morphological parameters are scaled up to full resolution. Supports preview-only mesh generation or mask export with force_new_label.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Root data directory." }, { "name": "filename", "type": "str", "default": "required", "description": "Subject filename or 'atlas'." }, { "name": "edit", "type": "str", "default": "optional", "description": "Edit NIfTI filename." }, { "name": "full_resolution", "type": "bool", "default": "false", "description": "Use full-resolution volume for processing." }, { "name": "preview_only", "type": "bool", "default": "true", "description": "When true with save_as_mask false, return GLB mesh preview only." }, { "name": "force_new_label", "type": "bool", "default": "false", "description": "When saving mask and one exists, allocate a new label instead of returning mask_already_exists." }, { "name": "save_as_mask", "type": "bool", "default": "false", "description": "Save binary result as segmentation mask instead of returning mesh." }, { "name": "low", "type": "float", "default": "0.10", "description": "Hysteresis low threshold on normalized intensities." }, { "name": "high", "type": "float", "default": "0.30", "description": "Hysteresis high threshold on normalized intensities." }, { "name": "closing_radius", "type": "int", "default": "3", "description": "Binary closing ball radius in voxels." }, { "name": "largest_component_mode", "type": "str", "default": "largest", "description": "Component filter: 'off', 'largest' (keep single largest CC), or 'above_min' (remove objects smaller than min_size)." }, { "name": "min_size", "type": "int", "default": "1000", "description": "Minimum connected-component volume in voxels for debris removal." }, { "name": "fill_mesh", "type": "bool", "default": "true", "description": "When true, march cubes on the closed/filled binary mask; when false, march cubes on raw hysteresis edges." } ], "algorithm": { "text": "1) Load volume and metadata via _load_volume_and_meta. 2) If save_as_mask and not preview_only and not full_resolution, scale closing_radius by resolution_factor and min_size by resolution_factor³. 3) _run_pipeline: normalize, hysteresis, binary closing, optional largest-component or above_min filtering. 4a) Mask path: write .nii.mask.gz with new or merged label; mirror to companion resolution. 4b) Mesh path: swap axes for Babylon, marching_cubes(level=0.5), center and normalize verts, decimate if >250k vertices, return base64 GLB with scale_factor and center.", "math": [ { "equation": "r' = r \\cdot f", "caption": "r: radius (or length) measured on the lossy preview; f: resolution_factor; r': full-resolution length." }, { "equation": "s' = s \\cdot f^{3}", "caption": "s: volume measured on lossy voxels; s': physical/full-res scaled volume (cubic in f)." } ], "diagram": "flowchart TD\n A[\"POST EdgeDetectionView\"] --> B[\"Load volume + spacing\"]\n B --> C[\"Scale params if lossy→full save\"]\n C --> D[\"Hysteresis + closing + CC filter\"]\n D --> E{\"save_as_mask?\"}\n E -->|\"yes\"| F[\"Write mask label\"]\n E -->|\"no\"| G[\"Marching cubes → GLB\"]\n F --> H[\"Write companion resolution mask\"]\n G --> I[\"Return gltf + scale + center\"]" }, "related": [ "edge-detection-slice-preview", "marchingcubes" ], "video": null } ] }, { "id": "resampling", "title": "Resampling", "methods": [ { "id": "deformation-based-mri-interpolator", "title": "Deformation-Based MRI Interpolator", "keywords": [ "resample", "anisotropic", "isotropic", "ants", "deformation", "mri" ], "summary": "Class that converts anisotropic MRI volumes to isotropic voxel spacing using bidirectional ANTs slice registration. Requires exactly one thick axis and two equal thin axes (within 1% tolerance). Used during scan extraction in views.py when anisotropy ratio exceeds 1.1. Default registration_type is SyNOnly for performance.", "options": [ { "name": "registration_type", "type": "str", "default": "SyNOnly", "description": "ANTs registration method: 'SyNOnly' (elastic only), 'SyN', 'Affine', or 'Rigid'." }, { "name": "verbose", "type": "bool", "default": "false", "description": "Enable progress and diagnostic print output." } ], "algorithm": { "text": "Constructor stores registration_type and verbose. Main entry is interpolate_anisotropic_to_isotropic (see related entry). Internally: auto-detect thick axis as argmax(voxel_size); validate single maximum and equal thin axes; compute target isotropic size as min(thin axes); upsample slice count by scale_factor = thick_size / target_size; for each adjacent slice pair precompute A→B and B→A ANTs SyN deformations; for each target slice position interpolate by scaled deformations and distance-weighted blend of moved A and B slices.", "math": [ { "equation": "f = s_{\\mathrm{thick}} / s_{\\mathrm{iso}}", "caption": "Scale factor from thick-slice spacing to isotropic (min in-plane) spacing; in-plane axes must agree within 1%." }, { "equation": "N' = \\mathrm{round}(N \\cdot f)", "caption": "N: original thick-axis slice count; N′: interpolated count along that axis." }, { "equation": "w_A = (p - p_A)/(p_B - p_A),\\quad w_B = 1 - w_A", "caption": "Same blend weights as interpolate_anisotropic_to_isotropic (paper §2.2.2)." }, { "equation": "I(p) = w_B\\, A_{\\mathrm{moved}} + w_A\\, B_{\\mathrm{moved}}", "caption": "Warped-neighbor blend after bidirectional SyNOnly between A and B." } ], "diagram": "flowchart TD\n A[\"DeformationBasedMRIInterpolator\"] --> B[\"interpolate_anisotropic_to_isotropic\"]\n B --> C[\"Detect thick axis + validate\"]\n C --> D[\"Precompute pair deformations ANTs SyN\"]\n D --> E[\"For each target slice position\"]\n E --> F[\"Scale + apply A→B / B→A fields\"]\n F --> G[\"Blend A_moved and B_moved\"]\n G --> H[\"Isotropic volume\"]" }, "related": [ "interpolate-anisotropic-to-isotropic", "interpolate-mri-bidirectional" ], "video": null }, { "id": "interpolate-anisotropic-to-isotropic", "title": "Interpolate Anisotropic to Isotropic", "keywords": [ "resample", "anisotropic", "isotropic", "deformation", "slice interpolation" ], "summary": "Converts anisotropic MRI (thick-slice) volumes to isotropic spacing by bidirectional ANTs SyNOnly registration between adjacent slices and distance-weighted blending of the warped neighbors (Ehrhardt-style), rather than simple trilinear smoothing.", "options": [ { "name": "image", "type": "np.ndarray", "default": "required", "description": "Input 3D MRI volume." }, { "name": "voxel_size", "type": "Tuple[float, float, float]", "default": "required", "description": "Voxel dimensions (x, y, z) in mm." }, { "name": "target_axis", "type": "int | None", "default": "null", "description": "Axis index (0, 1, or 2) to interpolate. When null, auto-selects the axis with largest voxel size." } ], "algorithm": { "text": "1) Identify the thick axis as argmax(voxel_size); require the two in-plane spacings to match within 1% tolerance; target spacing = min in-plane size. 2) For each adjacent pair (A, B), compute bidirectional SyNOnly fields (cross-correlation; reg_iterations = (200, 100, 100, 100), grad_step = 0.2, flow_sigma = 3, singleprecision = True). 3) For each intermediate physical position p, set weight_A = (p − p_A)/(p_B − p_A), weight_B = 1 − weight_A; warp A with the A→B field scaled by weight_A and B with B→A scaled by weight_B; blend I(p) = A_moved·weight_B + B_moved·weight_A (RegularGridInterpolator). 4) Stack intermediates into an isotropic volume.", "math": [ { "equation": "w_A = (p - p_A)/(p_B - p_A),\\quad w_B = 1 - w_A", "caption": "Fractional weights for target position p between adjacent original slices at p_A and p_B." }, { "equation": "I(p) = w_B\\, A_{\\mathrm{moved}} + w_A\\, B_{\\mathrm{moved}}", "caption": "Bidirectional blend of slices warped by fractionally scaled SyNOnly fields; avoids bias toward either neighbor." } ], "diagram": "flowchart TD\n A[\"3D anisotropic image\"] --> B[\"Validate axis anisotropy\"]\n B --> C[\"Compute new slice grid\"]\n C --> D[\"Register adjacent slice pairs ANTs\"]\n D --> E[\"Scale deformation fields by distance weight\"]\n E --> F[\"RegularGridInterpolator sample + blend\"]\n F --> G[\"Isotropic 3D output\"]" }, "related": [ "deformation-based-mri-interpolator", "interpolate-mri-bidirectional" ], "video": null, "images": [ { "caption": "Intermediate anatomy slices illustrate isotropic resampling of a thick-slice scan.", "path": "/static/docs-figures/docs_interpolate_anisotropic_to_isotropic.gif" } ] }, { "id": "interpolate-mri-bidirectional", "title": "Interpolate MRI Bidirectional", "keywords": [ "resample", "convenience", "anisotropic", "isotropic", "ants" ], "summary": "Module-level convenience wrapper that constructs a DeformationBasedMRIInterpolator and calls interpolate_anisotropic_to_isotropic, casting the result back to the input dtype. Default registration_type is 'SyN' (full SyN) and verbose is true, unlike the class default used in views.py.", "options": [ { "name": "image_array", "type": "np.ndarray", "default": "required", "description": "3D numpy array containing MRI data." }, { "name": "voxel_sizes", "type": "Tuple[float, float, float]", "default": "required", "description": "Tuple of (x, y, z) voxel sizes in mm." }, { "name": "registration_type", "type": "str", "default": "SyN", "description": "ANTs transform type: 'SyN' (recommended), 'Affine', or 'Rigid'." }, { "name": "verbose", "type": "bool", "default": "true", "description": "Enable progress output during interpolation." } ], "algorithm": { "text": "interpolator = DeformationBasedMRIInterpolator(registration_type, verbose); return interpolator.interpolate_anisotropic_to_isotropic(image_array, voxel_sizes).astype(image_array.dtype).", "math": "", "diagram": "flowchart TD\n A[\"interpolate_mri_bidirectional\"] --> B[\"Create DeformationBasedMRIInterpolator\"]\n B --> C[\"interpolate_anisotropic_to_isotropic\"]\n C --> D[\"Cast to input dtype\"]\n D --> E[\"Return isotropic array\"]" }, "related": [ "deformation-based-mri-interpolator", "interpolate-anisotropic-to-isotropic" ], "video": null } ] } ] }, { "id": "mesh-tools", "title": "Mesh Tools", "subcategories": [ { "id": "preserved-mesh-editing", "title": "Preserved Mesh Editing", "methods": [ { "id": "mesh-edit-mixin", "title": "Mesh Edit Mixin", "keywords": [ "preserved mesh", "ply edit", "landmarks", "metadata", "edit chain", "trimesh" ], "summary": "Shared backend mixin for preserved PLY scan edits. Resolves source meshes from the edit chain on disk (filename_edit_N_operation.ply), loads and validates Trimesh geometry, writes new edits through PreservedMeshEditWriter (which also refreshes quick-grid projection PNGs), and propagates landmark JSON plus distance sidecars whenever topology or coordinates change. Edit filenames are the source of truth; the scan JSON stores mesh summaries and elastic metadata but not a full edit list.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory containing extracted/." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem name (folder under extracted/)." }, { "name": "edit", "type": "string | null", "default": "latest", "description": "Specific PLY basename to use as input, or \"latest\" for the highest-numbered _edit_* PLY." } ], "algorithm": { "text": "1) Locate subject dir at extracted/{filename}/ and load {filename}.json metadata. 2) Resolve source PLY via edit argument or latest glob match on {filename}_edit_*_*.ply. 3) Load geometry with trimesh (concatenate multi-body scenes). 4) After an operation, allocate the next edit number, write {filename}_edit_{N}_{operation}.ply via PreservedMeshEditWriter.save, update landmarks for the new stem when present, and append mesh_summary to JSON metadata.", "math": "", "diagram": "flowchart TD\n A[\"HTTP ApplyMesh* request\"] --> B[\"Load metadata JSON\"]\n B --> C[\"Resolve source PLY edit\"]\n C --> D[\"Load Trimesh\"]\n D --> E[\"Run mesh operation\"]\n E --> F[\"PreservedMeshEditWriter.save\"]\n F --> G[\"Propagate landmarks\"]\n G --> H[\"Update scan JSON metadata\"]" }, "related": [ "preserved-mesh-edit-writer", "apply-mesh-crop", "apply-mesh-slice", "apply-mesh-rotation", "apply-mesh-cleanup", "alpaca-get-outer-mesh", "apply-mesh-shell", "apply-mesh-snap-to-mesh" ], "video": null }, { "id": "preserved-mesh-edit-writer", "title": "Preserved Mesh Edit Writer", "keywords": [ "ply export", "welded mesh", "projection png", "quick grid", "mesh grid" ], "summary": "Canonical export path for every preserved-mesh PLY edit. Welds coincident vertices so downstream tools see one connected surface, writes the PLY to disk, then regenerates front and back isometric projection PNGs used by the quick-access mesh grid. All ApplyMesh* endpoints and mesh elastic registration save through this writer.", "options": [ { "name": "mesh", "type": "trimesh.Trimesh", "default": "required", "description": "Mesh geometry to persist." }, { "name": "output_path", "type": "string", "default": "required", "description": "Absolute or relative path for the output .ply file." } ], "algorithm": { "text": "1) ensure_welded_trimesh merges duplicate vertex indices along shared edges. 2) Export PLY with trimesh. 3) Call ensure_mesh_projection_pngs(output_path, force=True) to render transparent front/back isometric PNGs beside the PLY.", "math": "", "diagram": "flowchart LR\n A[\"Trimesh\"] --> B[\"Weld vertices\"]\n B --> C[\"Export PLY\"]\n C --> D[\"ensure_mesh_projection_pngs\"]\n D --> E[\"Front + back PNG pair\"]" }, "related": [ "mesh-edit-mixin", "ensure-mesh-projection-pngs", "generate-mesh-grid-preview-jpeg" ], "video": null }, { "id": "ensure-mesh-projection-pngs", "title": "Ensure Mesh Projection PNGs", "keywords": [ "projection", "quick grid", "thumbnail", "vtk render", "preserved mesh" ], "summary": "Creates or refreshes the front ({stem}_projection.png) and back ({stem}_projection_back.png) thumbnail pair for a preserved-mesh PLY when missing or stale. Skips work when PNG mtimes are newer than the PLY unless force=True.", "options": [ { "name": "ply_path", "type": "string", "default": "required", "description": "Path to the preserved-mesh .ply file." }, { "name": "force", "type": "boolean", "default": "false", "description": "When true, regenerate PNGs even if they appear up to date." } ], "algorithm": { "text": "1) Compare PLY mtime against both projection PNGs; return early if current. 2) load_centered_preserved_mesh for display-normalized vertices. 3) Render isometric VTK screenshots (opposite view signs for front vs back). 4) Save composited RGBA PNGs next to the PLY.", "math": "", "diagram": "" }, "related": [ "preserved-mesh-edit-writer", "generate-mesh-grid-preview-jpeg" ], "video": null }, { "id": "generate-mesh-grid-preview-jpeg", "title": "Generate Mesh Grid Preview JPEG", "keywords": [ "grid preview", "jpeg", "quadrant", "axial", "sagittal", "coronal" ], "summary": "Builds a 2×2 JPEG mosaic (axial, isometric, sagittal, coronal orthographic renders) from a preserved-mesh PLY for richer grid or preview responses. Each quadrant is rendered at QUADRANT_RENDER_SIZE then pasted into a single composite image.", "options": [ { "name": "ply_path", "type": "string", "default": "required", "description": "Path to the preserved-mesh .ply file to preview." } ], "algorithm": { "text": "1) load_centered_preserved_mesh. 2) Render four orthographic/isometric VTK screenshots with fixed camera setups. 3) Resize each to a common quadrant size preserving aspect ratio. 4) Composite into a 2×2 RGB image and encode as JPEG bytes (GRID_PREVIEW_JPEG_QUALITY).", "math": "", "diagram": "flowchart TD\n PLY[\"PLY path\"] --> C[\"Centered mesh\"]\n C --> Q1[\"Axial\"]\n C --> Q2[\"Isometric\"]\n C --> Q3[\"Sagittal\"]\n C --> Q4[\"Coronal\"]\n Q1 --> G[\"2x2 JPEG mosaic\"]\n Q2 --> G\n Q3 --> G\n Q4 --> G" }, "related": [ "ensure-mesh-projection-pngs", "preserved-mesh-edit-writer" ], "video": null } ] }, { "id": "mesh-surface-utilities", "title": "Mesh surface utilities", "methods": [ { "id": "alpaca-get-outer-mesh", "title": "Outer shell extraction", "keywords": [ "outer shell", "outer surface", "visibility voting", "get_outer_mesh", "ALPACA.py", "apply-mesh-shell", "QuickMeshCanvas", "refresh menu", "Hidden Point Removal", "raycasting", "mesh tools" ], "summary": "Universal visibility-based outer-surface extractor implemented as ALPACA.get_outer_mesh in ALPACA.py. Keeps only externally visible triangles so internal cavities, nested marching-cubes sheets, and double walls cannot pollute downstream surface work. The helper lives in ALPACA.py for historical reasons, but it is not exclusive to the ALPACA rigid-alignment workflow — Aurora reuses it wherever a clean exterior shell is needed.\n\nPrimary UI entry: Quick mesh canvas cleanup/refresh menu → Outer shell (POST /apply-mesh-shell/), which persists _shell.ply (preserved mesh) or _shell.nii.gz (voxel). Also invoked when Align to reference runs with outer_surface=True, landmark snap / transfer with use_outer_shell, semi-landmark placement on the cranial shell, mesh elastic complex_mesh_volume / auto-extract presets, and related surface tools.", "options": [ { "name": "vertices", "type": "np.ndarray (N, 3)", "default": "required", "description": "Input mesh vertices (preserved PLY or marching-cubes surface from a voxel edit)." }, { "name": "faces", "type": "np.ndarray (M, 3)", "default": "required", "description": "Input mesh triangle indices." }, { "name": "invert_normals", "type": "bool", "default": "False", "description": "Flip winding / normals. Used automatically as a fallback when the first pass removes nearly all geometry (orientation was reversed)." } ], "algorithm": { "text": "Shared mesh utility (not an ALPACA-only alignment stage). Called from Apply Mesh Shell, Align to reference (outer_surface), snap/transfer paths (use_outer_shell), mesh elastic presets, semi-landmarks, and other surface tools:\n1) Weld coincident vertices so triangle adjacency is real (exported PLYs often duplicate verts).\n2) Multi-view visibility (method 4): sample viewpoints on a bounding sphere; for each triangle, cast a ray through its centroid and count views where it is the first hit (Open3D RaycastingScene).\n3) The vote histogram is typically bimodal — place the cutoff at the first valley so concavities visible from only some views are retained.\n4) Prune tiny connected components and heal small boundary holes; return cleaned vertices/faces.\n5) If reduction is extreme (>98% verts lost) and normals were not inverted, retry with invert_normals=True.\nOlder single-ray / normal-voting methods remain in ALPACA.py but are not selected.\n\n[[references]]", "math": [], "diagram": "flowchart TD\n A[\"PLY or voxel MC triangle mesh\"] --> B[\"Weld coincident verts\"]\n B --> C[\"Viewpoints on bounding sphere\"]\n C --> D[\"Per-triangle first-hit vote count\"]\n D --> E[\"Cutoff at first histogram valley\"]\n E --> F[\"Prune islands + heal holes\"]\n F --> G[\"Outer shell verts/faces\"]\n G --> H[\"Callers: Apply Mesh Shell · Align outer_surface · snap · elastic · semi-landmarks\"]", "references": [ { "id": "katz-hpr", "label": "Hidden Point Removal (inspiration)", "blurb": "Katz et al., 2007 — visibility idea; Aurora uses multi-view mesh voting instead of per-view HPR on clouds.", "library": "Literature", "symbol": "Katz et al., 2007", "url": "https://doi.org/10.1145/1275808.1276407" }, { "id": "open3d-raycast", "label": "Open3D RaycastingScene", "blurb": "First-hit ray queries used for multi-view triangle visibility votes.", "library": "Open3D", "symbol": "open3d.t.geometry.RaycastingScene", "url": "https://www.open3d.org/docs/latest/python_api/open3d.t.geometry.RaycastingScene.html" } ] }, "related": [ "apply-mesh-shell", "apply-mesh-snap-to-mesh", "align-to-reference", "alpaca-align-landmarks-to-mesh", "alpaca-create-landmarks-from-mesh", "get-semi-landmarks", "snap-to-mesh", "mesh-elastic-registration" ], "video": null, "images": [ { "caption": "Embryo marching-cubes surface sheds internal sheets until only the externally visible outer shell remains — the shared ALPACA.get_outer_mesh utility used by Apply Mesh Shell, Align to reference (outer_surface), and other Aurora surface tools.", "path": "/static/docs-figures/docs_alpaca_get_outer_mesh.gif" } ] } ] }, { "id": "mesh-edit-operations", "title": "Mesh Edit HTTP Operations", "methods": [ { "id": "apply-mesh-crop", "title": "Apply Mesh Crop", "keywords": [ "crop", "bounding box", "clip", "reorigin", "preserved mesh", "POST apply-mesh-crop" ], "summary": "Clips a preserved-mesh PLY to an axis-aligned bounding box and saves a new _cropped.ply edit. Expands the box by optional padding, re-originates surviving vertices so the crop minimum becomes the new origin, and drops landmarks outside the box while re-originating kept positions.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem to crop." }, { "name": "edit", "type": "string | null", "default": "null", "description": "Source PLY basename; omit to use metadata mesh_file or latest edit." }, { "name": "crop", "type": "number[6]", "default": "required", "description": "Two corners [x0,y0,z0,x1,y1,z1] in mesh mm; min/max are taken automatically." }, { "name": "padding", "type": "number", "default": "0", "description": "Extra mm added outward on every axis before clipping." } ], "algorithm": { "text": "1) Load source mesh. 2) Compute mins/maxs from crop corners ± padding. 3) Clip faces outside the box and shift vertices with reorigin_mesh_coords_after_mm_crop. 4) Write {filename}_edit_{N}_cropped.ply. 5) Filter and re-origin landmarks; copy subset of landmark distances.", "math": [ { "equation": "m = \\min(c_{0:3}, c_{3:6}) - p", "caption": "m: padded lower corner; c: crop-box corners; p: padding." }, { "equation": "M = \\max(c_{0:3}, c_{3:6}) + p", "caption": "M: padded upper corner of the retained axis-aligned bounds." }, { "equation": "v' = v - m + p", "caption": "After clipping, vertices are recentered into the padded crop frame." } ], "diagram": "" }, "related": [ "mesh-edit-mixin", "apply-mesh-slice" ], "video": null }, { "id": "apply-mesh-slice", "title": "Apply Mesh Slice", "keywords": [ "slice", "slicer", "isolate", "delete", "plane clip", "preview", "POST apply-mesh-slice" ], "summary": "Interactive slicer for preserved meshes and voxel volumes. Accepts a list of wedge operations (convex volumes defined by plane sets from volumes or outline rays). For preserved meshes, save=false returns a GLB preview without writing; save=true (or commit=true) writes _sliced.ply or _sliced.nii.gz depending on scan type. Landmarks are kept or removed according to the same wedge rules as geometry.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem." }, { "name": "edit", "type": "string | null", "default": "null", "description": "Source edit basename." }, { "name": "operations", "type": "object[]", "default": "required", "description": "Non-empty list of slicer operations (see action, volumes, outline_points, outline_rays)." }, { "name": "save", "type": "boolean", "default": "false", "description": "When true, persist a sliced edit. Aliased by commit." }, { "name": "commit", "type": "boolean", "default": "false", "description": "Truthy alias for save." }, { "name": "operations[].action", "type": "string", "default": "required", "description": "\"isolate\" keeps geometry inside any wedge; \"delete\" removes geometry inside wedges." }, { "name": "operations[].volumes", "type": "object[]", "default": "optional", "description": "Each volume supplies rays converted to plane sets; alternative to outline_points/outline_rays." }, { "name": "operations[].outline_points", "type": "number[][]", "default": "optional", "description": "Outline vertex indices paired with outline_rays when volumes is omitted." }, { "name": "operations[].outline_rays", "type": "object[]", "default": "optional", "description": "Ray definitions parallel to outline_points; triangulated into wedge plane sets." } ], "algorithm": { "text": "For each operation: build plane sets from volumes or triangulated outline rays. isolate: clip mesh to each wedge and concatenate survivors; delete: subtract convex volumes from the mesh. Repeat sequentially. Mesh path uses view-projected plane clipping; voxel path masks foreground voxels in physical mm. Preview mode returns decimated GLB via _mesh_preview_payload without writing files.", "math": [ { "equation": "p \\in W \\iff \\forall i:\\ (p - o_i)\\cdot n_i \\ge 0", "caption": "W: wedge kept by the slice; o_i, n_i: origin and inward normal of plane i." } ], "diagram": "flowchart TD\n R[\"operations list\"] --> A{\"action\"}\n A -->|\"isolate\"| I[\"Clip to wedge union\"]\n A -->|\"delete\"| D[\"Subtract wedges\"]\n I --> N[\"Next operation\"]\n D --> N\n N --> S{\"save?\"}\n S -->|\"no\"| P[\"GLB preview\"]\n S -->|\"yes\"| W[\"Write _sliced edit\"]" }, "related": [ "mesh-edit-mixin", "apply-mesh-crop" ], "video": null }, { "id": "apply-mesh-rotation", "title": "Apply Mesh Rotation", "keywords": [ "rotation", "babylon", "manual align", "preserved mesh", "POST apply-mesh-rotation" ], "summary": "Applies a manual rotation from the Babylon 3D viewer to a preserved-mesh PLY and saves _rotated.ply. Converts the Babylon 3×3 display matrix (Y/Z axis flip relative to stored PLY coordinates) into raw mesh space, rotates vertices about quick_mesh_properties.center (or surface centroid fallback), and rotates landmark positions identically.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem." }, { "name": "edit", "type": "string | null", "default": "null", "description": "Source PLY basename." }, { "name": "rotation", "type": "object", "default": "required", "description": "Babylon rotation payload; rotation._m must contain keys 0,1,2,4,5,6,8,9,10 forming a 3×3 matrix." } ], "algorithm": { "text": "1) Extract 3×3 matrix from rotation._m. 2) Map to raw mesh basis: R_raw = diag(1,−1,−1) · R_babylon · diag(1,−1,−1). 3) Rotate vertices: v' = (v − c) @ R_raw + c. 4) Apply same transform to landmarks. 5) Write _rotated.ply and store rotation matrices in edit metadata.", "math": [ { "equation": "v' = (v - c) R + c", "caption": "v: vertex; c: rotation_center from quick_mesh_properties; R: raw rotation matrix." } ], "diagram": "" }, "related": [ "mesh-edit-mixin", "apply-mesh-crop" ], "video": null }, { "id": "apply-mesh-cleanup", "title": "Apply Mesh Cleanup", "keywords": [ "cleanup", "connected components", "islands", "disconnected mesh", "POST apply-mesh-cleanup" ], "summary": "Removes disconnected mesh islands from a preserved PLY, keeping the largest surface-area component by default or all components above a minimum area fraction. Writes _cleaned.ply when components are removed; returns skipped status when the mesh is already a single component or cleanup preconditions fail (e.g. post-elastic edits).", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem." }, { "name": "edit", "type": "string", "default": "latest", "description": "Source PLY basename; defaults to latest edit." }, { "name": "use_island_volume_threshold", "type": "boolean", "default": "false", "description": "When true, keep every component whose surface area ≥ min_island_volume_percent of total area." }, { "name": "min_island_volume_percent", "type": "number", "default": "1.0", "description": "Minimum component area as percent of total surface area (clipped to 0.1–30)." } ], "algorithm": { "text": "1) Build face adjacency graph via scipy connected_components on welded edges. 2) Score each component by surface area. 3) Select keep_indices (largest only, or all above threshold). 4) Drop faces from removed components, remove_unreferenced_vertices. 5) Drop landmarks whose nearest face was removed. 6) Save _cleaned.ply.", "math": [ { "equation": "a_{\\min} = (p/100)\\, \\sum_i a_i", "caption": "a_min: area cutoff; p = min_island_volume_percent (default 1.0, clamped to 0.1–30); a_i: component surface areas; smaller islands are removed. The /100 converts percent to a fraction." } ], "diagram": "" }, "related": [ "mesh-edit-mixin", "apply-mesh-shell" ], "video": null }, { "id": "apply-mesh-shell", "title": "Apply Mesh Shell", "keywords": [ "outer shell", "get_outer_mesh", "apply-mesh-shell", "QuickMeshCanvas", "refresh menu", "complex volume", "POST apply-mesh-shell" ], "summary": "Quick mesh / HTTP entry that persists an outer-shell edit. Calls the shared Outer shell extraction algorithm (ALPACA.get_outer_mesh) — the same visibility utility used by Align to reference, landmark snap, mesh elastic, and other tools. UI: Quick mesh canvas cleanup/refresh menu → Outer shell.\n\nPreserved meshes: get_outer_mesh on the PLY → _shell.ply. Voxel scans: marching cubes at threshold, outer-shell extraction, rasterize to a thickened volume mask (~10 voxel dilation), and save _shell.nii.gz plus lossy preview. Landmarks far from the shell surface are dropped.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem." }, { "name": "edit", "type": "string | null", "default": "null", "description": "Source edit basename." }, { "name": "gaussian_blur", "type": "number | null", "default": "null", "description": "Optional Gaussian sigma applied to voxel volume before marching cubes (voxel path only)." } ], "algorithm": { "text": "1) Resolve source edit (preserved PLY or voxel NIfTI).\n2) Run shared Outer shell extraction (ALPACA.get_outer_mesh) — see that Mesh Tools entry for the visibility algorithm.\n3) Preserved mesh: trimesh cleanup → PreservedMeshEditWriter → _shell.ply.\n4) Voxel: optional Gaussian blur → marching cubes → outer shell → rasterize shell to binary mask → dilate ~10 iterations → keep original intensities inside mask → write full-res and lossy NIfTI edits.\n5) Drop landmarks far from the shell surface.", "math": "", "diagram": "flowchart TD\n U[\"Quick mesh Outer shell / POST apply-mesh-shell\"] --> M{\"preserved mesh?\"}\n M -->|\"yes\"| P[\"ALPACA.get_outer_mesh on PLY\"]\n M -->|\"no\"| V[\"MC + get_outer_mesh + rasterize\"]\n P --> S[\"Save _shell.ply\"]\n V --> T[\"Save _shell.nii.gz\"]\n S --> D[\"See Outer shell extraction for algorithm\"]\n T --> D" }, "related": [ "alpaca-get-outer-mesh", "apply-mesh-cleanup", "mesh-elastic-registration", "apply-mesh-snap-to-mesh", "align-to-reference" ], "video": null }, { "id": "apply-mesh-snap-to-mesh", "title": "Apply Mesh Snap to Mesh", "keywords": [ "snap", "landmarks", "closest point", "outer shell", "POST apply-mesh-snap-to-mesh" ], "summary": "Projects landmark positions onto the nearest point on the preserved-mesh surface (optionally the outer shell only), overwriting the scan's landmark JSON in place. Recomputes per-landmark distances and writes scan metadata statistics (mean/std, outlier counts vs voxel_size).", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "filename", "type": "string", "default": "required", "description": "Scan stem." }, { "name": "edit", "type": "string", "default": "latest", "description": "PLY edit whose stem selects {stem}_landmarks.json." }, { "name": "use_outer_shell", "type": "boolean", "default": "true", "description": "When true, snap against Outer shell extraction (ALPACA.get_outer_mesh) of the loaded PLY." } ], "algorithm": { "text": "1) Load mesh (outer shell optional). 2) Read landmarks for edit stem. 3) alpaca.calculate_landmarks_closest_points moves each landmark to the nearest triangle surface point. 4) Write updated landmarks JSON and {stem}_landmark_distances.json with snap_distance array. 5) Update metadata.landmarks summary fields.", "math": [ { "equation": "d_i = \\|p_i - q_i\\|_2", "caption": "d_i: snap distance; p_i: source point; q_i: closest point on the target surface." } ], "diagram": "" }, "related": [ "alpaca-get-outer-mesh", "mesh-edit-mixin", "apply-mesh-shell" ], "video": null } ] }, { "id": "mesh-elastic-registration-section", "title": "Elastic registration", "methods": [ { "id": "mesh-elastic-registration-mixin", "title": "Elastic registration (mesh pipeline)", "keywords": [ "NR-ICP", "nricp", "amberg", "preserved mesh", "elastic", "outer shell" ], "summary": "Shared Non-Rigid ICP implementation used by MeshElasticRegistrationView (and landmark-transfer helpers). Mirrors the user goals of voxel ElasticRegistrationView — warp subjects toward the reference — but keeps fixed PLY topology instead of storing ANTs deformation fields. Prefer reading this entry for stage weights, preset tables, propagation math, and homology/mask internals; the HTTP view entry covers batch I/O and eligibility.", "options": [ { "name": "mode", "type": "string", "default": "custom", "description": "Preset: \"facial\", \"complex_mesh_volume\", or \"custom\"." }, { "name": "mesh_elastic_custom_options.steps", "type": "number[3][]", "default": "[[0.01,0.5,10],…]", "description": "Custom mode only: up to 6 rows [stiffness, normal_weight, max_iter]. Landmark weight is always 0." }, { "name": "mesh_elastic_custom_options.gamma", "type": "number", "default": "1.0", "description": "Custom mode NR-ICP gamma (0.01–10)." }, { "name": "mesh_elastic_custom_options.eps", "type": "number", "default": "0.0001", "description": "Custom mode convergence epsilon (1e-6–1)." }, { "name": "distance_threshold", "type": "number", "default": "0.10", "description": "Custom mode correspondence cutoff in mm (0.01–0.5). Presets supply their own values." }, { "name": "use_vertex_normals", "type": "boolean", "default": "true", "description": "Enable normal terms in NR-ICP stages." }, { "name": "use_outer_shell", "type": "boolean", "default": "false", "description": "Extract outer shell on reference (and honor preset defaults)." }, { "name": "outer_shell_only", "type": "boolean", "default": "false", "description": "Register decimated outer shells; upsample displacements back to full topology." }, { "name": "auto_extract_subject_outer_shell", "type": "boolean", "default": "false", "description": "Persist _shell.ply before NR-ICP; requires outer_shell_only=true." }, { "name": "mask_subject_with_template", "type": "boolean", "default": "false", "description": "Warp reference template to subject ROI and crop subject geometry (facial preset option)." }, { "name": "mask_pad_centroid_fraction", "type": "number", "default": "0.04", "description": "Parallel-to-surface pad as a fraction of reference centroid size (0.005–0.20): max(|normal offset|, tangential exterior), a smooth along-surface ROI without the rounded isotropic rim tube." }, { "name": "tps_alpha_spacing_fraction", "type": "number", "default": "0.25", "description": "TPS regularization spacing fraction when TPS displacement propagation is used." } ], "algorithm": { "text": "Presets (MESH_ELASTIC_PRESETS):\n• facial — five stages with decreasing stiffness ws≈0.045→0.009, wn mostly 0.5 then 0, distance_threshold 0.08 mm; TPS upsample when decimated. Aimed at single-sheet facial surfaces.\n• complex_mesh_volume — five stages, distance_threshold 0.12 mm, outer_shell_only + use_outer_shell; k-NN upsample (TPS on closed skull shells can spike). Nested marching-cubes sheets without a shell confuse correspondences.\n• custom — user steps/gamma/eps plus request flags; defaults to a 4-stage schedule and 0.10 mm cutoff.\n\nSolver details: nricp_amberg with use_faces=False (memory-safe nearest vertex), wl forced to 0 (guidepoints only initialize via Procrustes; they are not soft constraints). Decimation cap MESH_ELASTIC_NRICP_VERTEX_CAP≈35000. After decimation, _sanitize_mesh_for_nricp merges vertices and drops disconnected debris that would singularize the LU factorization.\n\nHomology/mask branch: temporarily sets homology_mode and runs NR-ICP reference→subject, then _crop_mesh_by_surface_distance with a parallel-to-surface pad from mask_pad_centroid_fraction (default 4% of centroid size; max of normal and tangential offset). Poor template fit yields holes — check template_fit_within_pad_fraction in logs/edit metadata.\n\nPropagation: undecimated same-topology → direct; outer_shell_only → _nn_displacement_propagate_to_full_mesh (k=4 IDW); else → _tps_propagate_to_full_mesh with MESH_ELASTIC_TPS_ALPHA_SPACING_FRACTION=0.25. _assert_elastic_preserves_source_topology enforces vertex/face identity before save.", "math": [ { "equation": "E = w_s E_{\\mathrm{stiff}} + w_n E_{\\mathrm{normal}} + E_{\\mathrm{data}}", "caption": "NR-ICP stage energy with landmark weight w_l = 0 in this mixin path; w_s, w_n from the stage tuple." }, { "equation": "\\mathrm{score} = 1 - \\frac{1}{N}\\sum_i \\mathbb{I}(d_i > 6\\Delta)", "caption": "Surface agreement: d_i closest-point distance; Δ = voxel_size_mm; outliers beyond 6·Δ (fixed multiplier 6 in _mesh_elastic_error_threshold_mm)." }, { "equation": "t = \\bar{y} - s R \\bar{x}", "caption": "Umeyama similarity init from guidepoints: R, s from SVD (reflection-fixed); t aligns centroids." }, { "equation": "w_j \\propto 1 / \\max(\\|p - c_j\\|, \\varepsilon)", "caption": "IDW upsample weights from control points c_j to vertex p (default k_neighbors = 4); ε = 1e-10 distance floor." } ], "diagram": "flowchart TD\n A[\"_resolve_mesh_elastic_settings\"] --> B{\"mode\"}\n B -->|\"facial\"| C[\"Sheet-friendly steps, dt=0.08mm, TPS\"]\n B -->|\"complex_mesh_volume\"| D[\"Shell-only steps, dt=0.12mm, k-NN\"]\n B -->|\"custom\"| E[\"User steps/gamma/eps + flags\"]\n C --> F[\"_run_mesh_nricp\"]\n D --> F\n E --> F\n F --> G[\"Optional Procrustes from guidepoints\"]\n G --> H[\"Decimate + sanitize\"]\n H --> I[\"Staged nricp_amberg\"]\n I --> J{\"propagation\"}\n J -->|\"direct\"| K[\"Same topology\"]\n J -->|\"knn\"| L[\"Shell → full\"]\n J -->|\"tps\"| M[\"Decimated → full\"]\n K --> N[\"Topology assert + metrics\"]\n L --> N\n M --> N" }, "related": [ "mesh-elastic-registration", "load-mesh-elastic-reference-surface", "compute-mesh-to-reference-surface-distances" ], "video": null }, { "id": "mesh-elastic-registration", "title": "Elastic registration", "keywords": [ "batch elastic", "NR-ICP", "preserved mesh", "POST mesh-elastic-registration", "WebSocket progress" ], "summary": "Batch HTTP endpoint (POST /mesh-elastic-registration/) for Aurora’s mesh “Elastic registration” tool. It deforms every eligible preserved-mesh subject onto the reference surface with Non-Rigid ICP (trimesh nricp_amberg) while keeping fixed vertex topology — so per-vertex displacements, landmark transfer by index, and surface heatmaps remain well-defined.\n\nThis is the mesh half of the Elastic registration UI. Voxel projects instead call ElasticRegistrationView (ANTs SyN) on /elastic-registration/; mixed projects show both sections. Run after Align to reference: NR-ICP assumes subjects are already roughly co-located (rigid_prealignment is hard-coded off).\n\nPresets: facial (single-sheet surfaces), complex_mesh_volume (outer-shell-only NR-ICP + k-NN upsample), or custom. Subjects already carrying an *_elastic.ply edit are skipped until that edit is removed.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root." }, { "name": "reference", "type": "string", "default": "required", "description": "Reference scan stem or atlas." }, { "name": "mode", "type": "string", "default": "custom", "description": "facial | complex_mesh_volume | custom. UI always sends one of these (there is no backend balanced preset)." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "off | exclude | only for flagged subjects." }, { "name": "distance_threshold", "type": "number (mm)", "default": "0.10 custom; facial 0.08; complex 0.12", "description": "NR-ICP correspondence cutoff in absolute mm. UI slider labeled % sends value/100 as mm." }, { "name": "use_vertex_normals", "type": "boolean", "default": "true", "description": "Pass normals into nricp_amberg when enabled." }, { "name": "use_outer_shell / outer_shell_only", "type": "boolean", "default": "false (complex preset true)", "description": "Solve on outer shells; outer_shell_only keeps full input topology and upsamples via k-NN." }, { "name": "auto_extract_subject_outer_shell", "type": "boolean", "default": "false", "description": "Write a _shell.ply edit before NR-ICP; requires outer_shell_only." }, { "name": "mask_subject_with_template", "type": "boolean", "default": "false", "description": "Internal ref→subject NR-ICP + crop to a warped reference ROI before the final elastic pass." }, { "name": "mesh_elastic_custom_options", "type": "object", "default": "4-stage default steps", "description": "Custom mode: steps [[ws,wn,max_iter],…], gamma, eps (wl is always forced to 0)." } ], "algorithm": { "text": "Goal: non-rigidly warp each preserved-mesh subject onto the reference surface without changing vertex count or face connectivity.\n\n0) Resolve mode → settings (_resolve_mesh_elastic_settings). facial / complex_mesh_volume load preset steps and distance_threshold; custom merges mesh_elastic_custom_options. Reject auto_extract without outer_shell_only. Partition preserved-mesh scans; drop reference, faulty, existing *_elastic.ply, then flagFilter.\n\n1) Reference surface (once): load preserved PLY in registration-frame shared mm, or marching-cubes a voxel reference (axis swap, σ=1, threshold). Optionally extract an outer shell. Persist mesh_elastic_reference_ply on the reference JSON.\n\n2) Per subject — optional pre-branches.\n • Outer-shell auto-extract: write _shell.ply and filter landmarks to the shell.\n • Template mask: internal homology NR-ICP (reference → subject), crop by parallel-to-surface pad max(|n·offset|, tangential) = mask_pad_centroid_fraction × centroid_size, save _masked.ply.\n • Guidepoints (≥3 pairs): Umeyama/Procrustes similarity init only (not soft NR-ICP landmarks; wl=0).\n\n3) Final NR-ICP (subject → reference).\n Choose solver mesh: full input, use_outer_shell on both, or outer_shell_only (shell for solve, full topology for output). Decimate toward ~35k verts; sanitize (merge vertices, drop tiny components) to avoid singular LU in nricp_amberg. Run multi-stage Amberg NR-ICP with steps [ws, wl=0, wn, max_iter], absolute mm distance_threshold, use_faces=False (cKDTree nearest vertex).\n\n4) Upsample displacements back to full topology: direct copy if undecimated; k-NN IDW (k=4) when outer_shell_only; else thin-plate spline with α = 0.25 × median control spacing. Assert topology unchanged; save *_elastic.ply.\n\n5) Metrics: unsigned closest-point distances from deformed vertices to the reference surface; store elastic_surface_distance = 1 − fraction(d_i > 6×voxel_size_mm) and elastic_error = mean(d_i). Return 200 with success count, or 500 if every subject failed.", "math": [ { "equation": "E = w_s E_{\\mathrm{stiff}} + w_l E_{\\mathrm{lm}} + w_n E_{\\mathrm{n}} + E_{\\mathrm{data}}", "caption": "NR-ICP energy: stiffness, landmark, normal, and data terms; correspondences with d > distance_threshold get zero data weight (preset cutoffs: facial 0.08 mm, complex_mesh_volume 0.12 mm, custom default 0.10 mm)." }, { "equation": "\\tau = 6\\Delta", "caption": "τ = 6·Δ with fixed multiplier 6 from _mesh_elastic_error_threshold_mm; Δ = voxel_size_mm (mesh-reference spacing)." }, { "equation": "\\mathrm{score} = 1 - \\frac{1}{N}\\sum_i \\mathbb{I}(d_i > \\tau)", "caption": "elastic_surface_distance; elastic_error is the mean of d_i." }, { "equation": "w_j \\propto 1 / \\max(\\|p - c_j\\|, \\varepsilon)", "caption": "k-NN / IDW upsample from deformed controls c_j (default k_neighbors = 4); ε = 1e-10 floor on distance in 1/max(∥p−c_j∥, ε)." }, { "equation": "\\alpha = 0.25 \\cdot \\mathrm{median\\, NN}(c)", "caption": "α = tps_alpha_spacing_fraction · median NN spacing among controls; default fraction MESH_ELASTIC_TPS_ALPHA_SPACING_FRACTION = 0.25 (UI option tps_alpha_spacing_fraction)." } ], "diagram": "flowchart TD\n A[\"POST /mesh-elastic-registration/\"] --> B[\"Resolve mode: facial / complex_mesh_volume / custom\"]\n B --> C[\"Filter preserved-mesh subjects\"]\n C --> D[\"Load + persist reference surface\"]\n D --> E[\"For each subject\"]\n E --> F{\"auto outer shell?\"}\n F -->|\"yes\"| G[\"Save _shell.ply\"]\n F -->|\"no\"| H[\"Optional guidepoint Procrustes\"]\n G --> H\n H --> I{\"mask_subject_with_template?\"}\n I -->|\"yes\"| J[\"Internal NR-ICP ref→subject + crop → _masked.ply\"]\n I -->|\"no\"| K[\"Final NR-ICP subject→reference\"]\n J --> K\n K --> L{\"solver mesh\"}\n L -->|\"outer_shell_only\"| M[\"NR-ICP on shell\"]\n L -->|\"use_outer_shell\"| N[\"Shell on subject + reference\"]\n L -->|\"full\"| O[\"Full registration input\"]\n M --> P[\"Decimate ~35k + sanitize\"]\n N --> P\n O --> P\n P --> Q[\"Multi-stage nricp_amberg wl=0\"]\n Q --> R{\"upsample\"}\n R -->|\"direct\"| S[\"Copy vertices\"]\n R -->|\"outer_shell_only\"| T[\"k-NN displacement\"]\n R -->|\"decimated\"| U[\"TPS to full mesh\"]\n S --> V[\"Assert topology; save _elastic.ply + metrics\"]\n T --> V\n U --> V\n V --> E\n E --> W[\"200 with registered count\"]" }, "related": [ "mesh-elastic-registration-mixin", "mesh-elastic-surface-metric", "load-mesh-elastic-reference-surface", "align-to-reference", "elastic-registration", "alpaca-get-outer-mesh" ], "video": null, "images": [ { "caption": "Cyan reference warps onto the red subject (homology), a surface-distance pad masks the ROI, then the masked subject elastically settles onto the reference — ending with excess · elastic · reference.", "path": "/static/docs-figures/docs_mesh_elastic_registration.gif" } ], "outputs": [ { "name": "message", "type": "string", "description": "Mesh elastic registration complete for X/Y subject(s); optional warnings array." }, { "name": "{scan}_edit_{n}_elastic.ply", "type": "file", "description": "Topology-preserving elastically deformed mesh." }, { "name": "{scan}_edit_{n}_shell.ply / _masked.ply", "type": "file", "description": "Optional intermediate edits when auto-shell or template-mask branches run." }, { "name": "mesh_elastic_reference_ply", "type": "metadata + file", "description": "Persisted reference surface used for registration and later heatmaps." }, { "name": "elastic_to / elastic_surface_distance / elastic_error", "type": "JSON fields", "description": "Reference name; fraction of vertices within τ=6×voxel_size_mm; mean surface distance in mm." } ] }, { "id": "load-mesh-elastic-reference-surface", "title": "Load Mesh Elastic Reference Surface", "keywords": [ "reference surface", "mesh_elastic_reference.ply", "marching cubes", "registration heatmap" ], "summary": "Loads the reference triangle mesh used for mesh-elastic metrics and registration heatmaps. Resolution order: mesh_elastic_reference_ply from JSON (written at registration), preserved-mesh mesh_file PLY, dedicated voxel reference PLY basename, or marching-cubes fallback from the latest full-res NIfTI. Preserved-mesh results are mapped into the registration frame via mesh_elastic_reference_in_registration_frame.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "reference", "type": "string", "default": "required", "description": "Reference scan stem or \"atlas\"." } ], "algorithm": { "text": "1) Open reference JSON (atlas.json or {reference}.json). 2) Try mesh_elastic_reference_ply on disk. 3) Else try preserved mesh_file PLY. 4) Else try {reference}_mesh_elastic_reference.ply. 5) Else marching cubes latest NIfTI with Gaussian σ=1, threshold from metadata, axes 0↔2 swap, reversed face winding.", "math": "", "diagram": "" }, "related": [ "prepare-heatmap-reference-mesh", "compute-mesh-to-reference-surface-distances", "mesh-elastic-registration" ], "video": null }, { "id": "prepare-heatmap-reference-mesh", "title": "Prepare Heatmap Reference Mesh", "keywords": [ "heatmap", "decimation", "vertex cap", "cache", "dense distances" ], "summary": "Builds a display-budget reference mesh for per-vertex heatmap coloring on preserved-mesh subjects. The full-resolution mesh_elastic_reference.ply stays on disk for landmark propagation; this function decimates to vertex_cap (matching the subject display budget) and caches vertices/faces as NPZ beside the source PLY for fast reuse in MarchingCubesView dense distance computation.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root directory." }, { "name": "reference", "type": "string", "default": "required", "description": "Elastic reference stem (elastic_to)." }, { "name": "vertex_cap", "type": "integer", "default": "required", "description": "Maximum reference vertices for heatmap queries (same cap as subject display mesh)." } ], "algorithm": { "text": "1) Resolve full reference via load path or marching-cubes fallback. 2) Return cached NPZ if vertex_cap and source mtime match. 3) Decimate full mesh to vertex_cap with _decimate_trimesh_to_vertex_cap. 4) Save {cache_stem}_heatmap_ref_v{cap}_vertices.npz and faces NPZ. 5) Return in-memory Trimesh for distance queries.", "math": "", "diagram": "flowchart LR\n Full[\"Full reference PLY\"] --> Dec[\"Decimate to vertex_cap\"]\n Dec --> Cache[\"NPZ cache\"]\n Cache --> Q[\"Open3D closest-point queries\"]" }, "related": [ "compute-mesh-to-reference-surface-distances", "load-mesh-elastic-reference-surface" ], "video": null }, { "id": "compute-mesh-to-reference-surface-distances", "title": "Compute Mesh to Reference Surface Distances", "keywords": [ "surface distance", "Open3D", "RaycastingScene", "heatmap", "closest point" ], "summary": "Computes unsigned Euclidean distance from each query vertex to the nearest point on a reference triangle mesh using Open3D RaycastingScene (same approach as MarchingCubesView dense_distances for voxels). Used for preserved-mesh heatmap coloring, elastic surface metrics, and template-mask quality checks. Processes vertices in chunks for memory efficiency.", "options": [ { "name": "query_vertices", "type": "np.ndarray (N, 3)", "default": "required", "description": "Subject vertex positions in registration mm." }, { "name": "reference_mesh", "type": "trimesh.Trimesh", "default": "required", "description": "Reference surface mesh (often decimated for heatmaps)." }, { "name": "label", "type": "string | null", "default": "null", "description": "Optional log label for timing messages." }, { "name": "chunk_size", "type": "integer", "default": "100000", "description": "Number of query vertices per Open3D batch." } ], "algorithm": { "text": "1) Build Open3D TriangleMesh from reference vertices/faces. 2) Create RaycastingScene and add triangles. 3) For each chunk of query_vertices, call compute_closest_points. 4) Store ||query − closest||₂ per vertex.", "math": [ { "equation": "d_i = \\min_{q \\in S} \\|v_i - q\\|_2", "caption": "Unsigned closest-point distance from vertex v_i to reference surface S." } ], "diagram": "" }, "related": [ "prepare-heatmap-reference-mesh", "mesh-elastic-surface-metric" ], "video": null }, { "id": "mesh-elastic-surface-metric", "title": "Mesh Elastic Surface Metric", "keywords": [ "elastic score", "surface distance", "error threshold", "registration quality" ], "summary": "Summarizes a distance array into the scalar scores stored on elastic subjects: fraction of vertices above threshold, elastic_surface_distance (1 minus that fraction), and mean distance in mm. Threshold is typically 6× reference voxel size, matching voxel ElasticRegistrationView surface scoring.", "options": [ { "name": "distances", "type": "np.ndarray", "default": "required", "description": "Per-vertex distances to reference surface in mm." }, { "name": "error_threshold_mm", "type": "number", "default": "required", "description": "Acceptance threshold in mm (usually 6 × voxel_size)." } ], "algorithm": { "text": "above_fraction = count(d > threshold) / N; surface_score = 1 − above_fraction; mean_distance = mean(d). Empty input returns (0, 0, 0).", "math": [ { "equation": "\\mathrm{score} = 1 - \\frac{1}{N}\\sum_i \\mathbb{I}(d_i > \\tau)", "caption": "Fraction of vertices within tolerance τ of the reference surface; τ = error_threshold_mm, typically 6·voxel_size_mm." }, { "equation": "\\overline{d} = \\frac{1}{N}\\sum_i d_i", "caption": "Mean closest-point distance (elastic_error)." } ], "diagram": "" }, "related": [ "compute-mesh-to-reference-surface-distances", "mesh-elastic-registration" ], "video": null } ] } ] }, { "id": "rigid-elastic-registration", "title": "Rigid & Elastic Registration", "subcategories": [ { "id": "rigid-batch-alignment", "title": "Align to reference", "methods": [ { "id": "align-to-reference", "title": "Align to reference", "keywords": [ "rigid alignment", "reference scan", "ALPACA", "ANTs", "guidepoints", "batch", "preserved mesh" ], "summary": "Rigidly aligns every eligible moving scan in a project into the chosen reference’s coordinate frame (rotation + translation, with optional uniform scaling). This is Aurora’s Step 2 “Align to reference” tool — the geometric normalization step that should run before intensity matching and elastic registration so anatomy is co-located across subjects.\n\nThree methods achieve the same goal by different evidence:\n• ALPACA (default) — surface point-cloud matching (FPFH + RANSAC + ICP) on meshes extracted from voxels or on preserved PLY meshes; works for mixed mesh/voxel projects.\n• ANTs rigid — intensity-based ANTs Rigid registration on voxel volumes only (not preserved meshes).\n• Manual guidepoints — Kabsch/Procrustes fit from homologous guidepoint pairs on reference and subject (voxels and preserved meshes).\n\nEligible movers exclude the reference, faulty scans, subjects already ending in an *_aligned edit, and subjects that already completed elastic registration. Progress streams over the progress WebSocket.\n\nWhen method=alpaca, Poisson sampling and FPFH → RANSAC → ICP are documented under ALPACA rigid alignment. Optional outer_surface prep uses the shared Outer shell extraction utility under Mesh Tools (also available from the Quick mesh Outer shell edit via Apply Mesh Shell).", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root containing extracted/ (and atlas/ when reference is atlas)." }, { "name": "reference", "type": "string", "default": "required", "description": "Reference scan stem, or atlas for the project atlas." }, { "name": "method", "type": "string", "default": "alpaca", "description": "Alignment evidence: alpaca | ants | manual-guidepoints (case-insensitive). UI labels: ALPACA*, ANTs rigid, Manual guidepoints." }, { "name": "scaling", "type": "boolean", "default": "false", "description": "If true, estimate a uniform scale so subject size matches the reference before/during the rigid fit (formula differs by method)." }, { "name": "outer_surface", "type": "boolean", "default": "false (UI default true)", "description": "When true (ALPACA path), run shared Outer shell extraction (ALPACA.get_outer_mesh) on reference and subject surfaces before Poisson sampling / FPFH. Same utility as Apply Mesh Shell / Quick mesh Outer shell." }, { "name": "interpolation", "type": "string", "default": "linear", "description": "Volume warp order for ALPACA/guidepoints rotation: linear → order 1, nearest → order 0." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "Batch subject filter: off | exclude | only (flagged scans)." }, { "name": "ants_rigid_options", "type": "object", "default": "see normalize_ants_rigid_options", "description": "ANTs-only nested knobs: aff_metric, aff_iterations, aff_shrink_factors, aff_smoothing_sigmas, aff_sampling, aff_random_sampling_rate, grad_step, use_histogram_matching, singleprecision." } ], "algorithm": { "text": "Goal: place every moving subject into the reference canvas so later elastic registration and morphometrics compare homologous anatomy.\n\n0) Parse directory, reference, method, scaling, outer_surface, interpolation, flagFilter, and ants_rigid_options. Build the mover list from extracted/, drop the reference, faulty scans, already-aligned latest edits, and elastic-completed subjects; apply flagFilter. Partition movers into preserved PLY vs voxel. ANTs is rejected (400) if the reference is a preserved mesh or no voxel movers remain.\n\n1) Load reference.\n • Preserved PLY reference: load latest PLY edit; build an imaginary voxel prism (resolve_mesh_reference_prism) so voxel movers can embed into a shared mm canvas.\n • Voxel reference: load latest NIfTI edit, Gaussian-smooth (σ=1), read threshold/spacing. For ALPACA/guidepoints, marching-cubes the reference surface (optional outer shell / decimation).\n\n2) Per moving scan — preserved PLY path (ALPACA or manual-guidepoints only).\n Re-express the subject in the reference prism when needed. ALPACA: align_landmarks_to_mesh → compose_source_to_target_transform → apply_rigid_transform on vertices (and guidepoints). Manual: Kabsch from homologous guidepoints with optional radial scale. Save *_aligned.ply and update mesh metadata; when the reference is voxel-based, apply the X↔Z display-basis swap for Babylon convention.\n\n3) Per moving scan — voxel NIfTI path (all three methods).\n Common prep: load latest edit, estimate background below threshold, resample to reference_voxel_size via zoom_factor = subject_vs / reference_vs.\n • ALPACA branch: marching-cubes the subject; optional outer shell and scale_match from mean radii; FPFH + multi-attempt RANSAC with burn-in scoring, then point-to-plane ICP (fallback to RANSAC if ICP worsens the score). Convert the rigid/similarity transform to voxel units; pad → rotate (ndimage.affine_transform with Rᵀ) → translate; embed by centroid into the reference-shaped canvas; strip a 20-voxel border to reduce later elastic edge artifacts; store_alignment_embedding for landmarks.\n • ANTs branch (voxel only): optional mesh-radius scaling; integer centroid pre-shift of binary masks; min_max_normalize to [−1,1]; ants.registration Rigid at half resolution with ants_rigid_options; ants.apply_transforms at full resolution; restore_range; border strip. Landmark transform for ANTs is not implemented yet.\n • Manual-guidepoints branch: require matching-shape guidepoint JSON on reference and subject; optional guidepoint radial scaling; Kabsch R; pad-rotate-translate; canvas embed + border strip.\n\n4) Save & propagate: write *_aligned.nii.gz with the reference affine; update alignment_* metadata; transform_landmarks_after_alignment for ALPACA/guidepoints; optional lossy NIfTI; replay the same chain on paired masks (nearest). Per-scan failures are logged and skipped so the batch can partially succeed.\n\n5) Return 200 when the batch finishes; WebSocket reports Alignment complete for all scans.", "math": [ { "equation": "z = \\delta_{\\mathrm{subj}} / \\delta_{\\mathrm{ref}}", "caption": "Resample zoom: subject voxel size over reference voxel size." }, { "equation": "s = \\mathbb{E}\\|p^{\\mathrm{ref}} - \\bar{p}^{\\mathrm{ref}}\\| / \\mathbb{E}\\|p^{\\mathrm{subj}} - \\bar{p}^{\\mathrm{subj}}\\|", "caption": "Optional ALPACA scale from mean landmark radius of reference vs subject." }, { "equation": "H = X_c^\\top Y_c = U\\Sigma V^\\top,\\quad R = V^\\top C U^\\top", "caption": "Kabsch: cross-covariance of centered landmarks; C = diag(1,1,±1) fixes reflections." }, { "equation": "v' = (v - \\bar{x}) R + \\bar{y}", "caption": "Apply rigid map from subject centroid x̄ to reference centroid ȳ." }, { "equation": "x_{\\mathrm{out}} = R^\\top x_{\\mathrm{in}} + (c - R^\\top c)", "caption": "Volume rotation via scipy uses Rᵀ about padded center c." } ], "diagram": "flowchart TD\n A[\"POST /align-to-reference/\"] --> B[\"Parse method, scaling, outer_surface, interpolation, flagFilter, ants_rigid_options\"]\n B --> C[\"Build movers: drop reference, faulty, already aligned, elastic-done\"]\n C --> D[\"apply_flag_filter\"]\n D --> E{\"method?\"}\n E -->|\"ants\"| F{\"preserved-mesh reference or no voxel movers?\"}\n F -->|\"yes\"| G[\"400: ANTs is voxel-only\"]\n F -->|\"no\"| H[\"Continue\"]\n E -->|\"alpaca / manual-guidepoints\"| H\n H --> I{\"Load reference\"}\n I -->|\"preserved PLY\"| J[\"PLY + imaginary prism canvas\"]\n I -->|\"voxel NIfTI\"| K[\"Latest edit + optional marching cubes\"]\n J --> L[\"For each moving scan\"]\n K --> L\n L --> M{\"Moving type?\"}\n M -->|\"preserved PLY\"| N{\"method?\"}\n N -->|\"alpaca\"| O[\"ALPACA mesh rigid; save *_aligned.ply\"]\n N -->|\"manual-guidepoints\"| P[\"Kabsch on guidepoints; save *_aligned.ply\"]\n N -->|\"ants\"| SKIP1[\"Skip preserved mesh\"]\n M -->|\"voxel NIfTI\"| Q[\"Resample to reference spacing; estimate background\"]\n Q --> R{\"method?\"}\n R -->|\"alpaca\"| S[\"MC surface; FPFH/RANSAC/ICP; pad-rotate-translate; embed canvas\"]\n R -->|\"ants\"| T[\"Centroid shift; normalize; ANTs Rigid half-res; apply full-res\"]\n R -->|\"manual-guidepoints\"| U{\"Matching guidepoints?\"}\n U -->|\"no\"| SKIP2[\"Skip scan\"]\n U -->|\"yes\"| V[\"Optional scale; Kabsch; pad-rotate-translate; embed\"]\n S --> W[\"Metadata + landmarks (ALPACA/GP) + optional lossy/mask\"]\n T --> W\n V --> W\n O --> X[\"Next scan\"]\n P --> X\n SKIP1 --> X\n SKIP2 --> X\n W --> X\n X --> L\n L --> Y[\"200: Alignment complete\"]" }, "related": [ "alpaca-align-landmarks-to-mesh", "mesh-elastic-registration", "invert-alignment-landmarks", "export-inverted-landmarks", "registration-tools-overview", "elastic-registration", "alpaca-get-outer-mesh", "alpaca-create-landmarks-from-mesh", "normalize-ants-rigid-options" ], "video": null, "images": [ { "caption": "ALPACA rigid align: misaligned clouds → RANSAC → ICP, then volume mid-slices confirming a shared grid.", "path": "/static/docs-figures/docs_align_to_reference.gif" } ], "outputs": [ { "name": "message", "type": "string", "description": "HTTP 200 summary, e.g. Alignment complete using {method} method." }, { "name": "{scan}_edit_{n}_aligned.nii.gz / .ply", "type": "file", "description": "Per-subject aligned volume (voxel path) or preserved mesh (PLY path)." }, { "name": "{scan}_lossy_edit_{n}_aligned.nii.gz", "type": "file", "description": "Optional lossy preview when lossy_compression is configured (voxel)." }, { "name": "{scan}_edit_{n}_aligned_landmarks.json", "type": "file", "description": "Landmarks propagated through the rigid embedding (ALPACA and manual-guidepoints only; ANTs landmark transform not implemented)." }, { "name": "{scan}_edit_{n}_aligned.nii.mask.gz", "type": "file", "description": "Paired mask replayed with the same rigid chain when a source mask exists." }, { "name": "alignment_* metadata", "type": "JSON fields", "description": "alignment_to, alignment_method, alignment_scale, alignment_calculated_parameters (rotation, centers, padding), voxel_size → reference spacing; ALPACA also writes alignment_score / alignment_error / alignment_median_nn." } ] } ] }, { "id": "alpaca-rigid-alignment", "title": "ALPACA rigid alignment", "methods": [ { "id": "alpaca-create-landmarks-from-mesh", "title": "Poisson-disk sampling", "keywords": [ "Poisson disk", "pseudo-landmarks", "point cloud", "ALPACA", "FPFH", "ALIGNMENT_REFERENCE_POISSON_POINTS", "Open3D" ], "summary": "ALPACA stage that builds evenly spaced surface point clouds for FPFH feature matching. The reference always uses ALIGNMENT_REFERENCE_POISSON_POINTS = 5000 samples; the subject uses a vertex-ratio-proportional count so neighborhood statistics stay comparable on both preserved meshes and voxel-derived marching-cubes shells.", "options": [ { "name": "vertices", "type": "np.ndarray (N, 3)", "default": "required", "description": "Mesh vertex coordinates in world space." }, { "name": "faces", "type": "np.ndarray (M, 3)", "default": "required", "description": "Triangle indices into vertices." }, { "name": "n_landmarks", "type": "int", "default": "0", "description": "Number of landmarks to sample. When 0, uses ALIGNMENT_REFERENCE_POISSON_POINTS so density matches the alignment reference path." } ], "algorithm": { "text": "1) Convert vertices/faces to an Open3D TriangleMesh and compute vertex normals.\n2) If n_landmarks is 0, use ALIGNMENT_REFERENCE_POISSON_POINTS = 5000.\n3) Inside the alignment pipeline the subject count is max(1, round(5000 × |V_subject| / |V_reference|)).\n4) sample_points_poisson_disk returns approximately uniform, non-clustering points.\n5) Alignment later builds low (1/5) / medium (1/3) / full resolution tiers from these clouds for coarse-to-fine ICP.", "math": [ { "equation": "n_{\\mathrm{ref}} = 5000", "caption": "Fixed reference Poisson budget (ALIGNMENT_REFERENCE_POISSON_POINTS)." }, { "equation": "n_{\\mathrm{subj}} = \\max\\!\\bigl(1,\\,\\mathrm{round}(5000\\cdot |V_{\\mathrm{subj}}|/|V_{\\mathrm{ref}}|)\\bigr)", "caption": "Subject sample count scales with mesh vertex ratio so point density (and FPFH support) stays matched." } ], "diagram": "flowchart TD\n A[\"Mesh verts/faces\"] --> B[\"Open3D TriangleMesh + normals\"]\n B --> C{\"n_landmarks?\"}\n C -->|\"0 / default\"| D[\"n_ref = 5000\"]\n C -->|\"set\"| E[\"Use requested n\"]\n D --> F[\"Poisson-disk sample\"]\n E --> F\n F --> G[\"Point cloud for FPFH / ICP\"]" }, "related": [ "alpaca-get-outer-mesh", "alpaca-align-landmarks-to-mesh", "align-to-reference" ], "video": null, "images": [ { "caption": "Poisson-disk samples appear on the embryo outer shell — the same ALPACA.create_landmarks_from_mesh surface sampler used for FPFH matching.", "path": "/static/docs-figures/docs_alpaca_create_landmarks_from_mesh.gif" } ] }, { "id": "alpaca-align-landmarks-to-mesh", "title": "ALPACA rigid alignment (FPFH · RANSAC · ICP)", "keywords": [ "ALPACA", "FPFH", "RANSAC", "ICP", "rigid alignment", "point cloud", "Otsu", "stochastic", "symmetry", "Porto", "scan-to-reference" ], "summary": "Aurora’s primary surface-based rigid alignment: an adaptation of ALPACA (Porto et al., 2021) that keeps only the rigid stages — FPFH matching, a stochastic multi-candidate RANSAC pool, symmetry disambiguation, and point-to-plane ICP. Landmark transfer via Coherent Point Drift from the original ALPACA is not used; homology for landmarks comes later from elastic voxel fields. Align to reference calls this path when method=alpaca (default). Optional outer_surface prep uses the shared Outer shell extraction utility under Mesh Tools, not an ALPACA-only stage.", "options": [ { "name": "source_vertices / source_faces", "type": "np.ndarray", "default": "required", "description": "Reference mesh geometry that will be moved." }, { "name": "target_vertices / target_faces", "type": "np.ndarray", "default": "required", "description": "Subject mesh geometry that stays fixed." }, { "name": "target_metadata_path", "type": "str", "default": "required", "description": "Path to the subject's metadata JSON where alignment success/parameters are recorded." }, { "name": "scaling_mode", "type": "bool", "default": "False", "description": "If true, allow a similarity (rigid + uniform scale) fit instead of pure rigid." }, { "name": "outer_surface", "type": "bool", "default": "False", "description": "When true, preprocess reference and subject with shared Outer shell extraction (ALPACA.get_outer_mesh / Mesh Tools) before Poisson sampling." } ], "algorithm": { "text": "Aurora adopts ALPACA’s rigid pipeline only (no CPD landmark transfer).\n\n1) Optional outer shell — if outer_surface, call the shared Outer shell extraction utility (Mesh Tools / ALPACA.get_outer_mesh; same as Apply Mesh Shell) on reference and subject.\n2) Poisson-disk clouds — reference n_ref = 5000; subject scaled by vertex ratio (see Poisson-disk sampling). Optional scaling_mode equalizes mean centroid distances first.\n3) Center + RMS-normalize — subtract centroids; divide by the reference cloud’s RMS radius so distance thresholds are scale-agnostic.\n4) Geometry-adaptive FPFH — map surface smoothness (median normal angular spread) linearly to a feature radius between 8% and 15% of the bounding-box diagonal; compute FPFH descriptors; keep bidirectional matches (unless too few), then edge-length-validate correspondences.\n5) Stochastic RANSAC pool — run n_ransac_attempts = 40 candidates, each with independent parameter jitter. Record per-reference-point NN distances → an N×40 matrix.\n6) Otsu selection — threshold all pooled distances (Otsu) into peak vs tail; rank candidates by composite score = median × (1 + IQR) / normal_consistency (lower is better).\n7) Homology mask — exclude reference points that fall in the Otsu tail for ≥90% of the top-10 ranked tries (crop edges / non-overlap).\n8) Symmetry check — test four 180° axis flips; keep the one maximizing normal consistency.\n9) Point-to-plane ICP — refine on the masked cloud (coarse→medium→full tiers). Discard ICP if the post-ICP composite score is worse than pre-ICP (partial-overlap safeguard).\n10) Map the 4×4 transform back to physical coordinates (undo RMS scale + centroids); write subject metadata / return the subject→reference mapping for Align to reference to apply to voxels, masks, and landmarks.\n\n[[references]]", "math": [ { "equation": "r_{\\mathrm{FPFH}} = \\bigl(0.08 + 0.07\\cdot s\\bigr)\\, L_{\\mathrm{diag}}", "caption": "Geometry-adaptive FPFH radius; s ∈ [0,1] from median normal angular spread (smoother → smaller radius); L_diag = reference cloud bounding-box diagonal. Code maps smoothness onto the 8%–15% band." }, { "equation": "\\mathrm{score} = \\frac{\\mathrm{median}(\\mathbf{d})\\cdot\\bigl(1+\\mathrm{IQR}(\\mathbf{d})\\bigr)}{\\mathrm{normal\\_consistency}}", "caption": "Composite ranking over the 40 RANSAC candidates; d = per-point NN distances after that try; lower score is better (tight low distances + high normal agreement)." }, { "equation": "\\tilde{X} = (X - c_s)/s,\\quad \\tilde{Y} = (Y - c_t)/s", "caption": "Centroid-centered, RMS-normalized clouds; s = RMS radius of the reference cloud." }, { "equation": "R'=R,\\quad t' = s\\, t + c_t - R\\, c_s", "caption": "Lift the normalized rigid (or similarity) transform back to physical coordinates." } ], "diagram": "flowchart TD\n A[\"Reference + subject meshes\"] --> B{\"outer_surface?\"}\n B -->|yes| C[\"Outer shell extraction (Mesh Tools)\"]\n B -->|no| D[\"Full meshes\"]\n C --> E[\"Poisson-disk sampling\"]\n D --> E\n E --> F[\"Center + RMS normalize\"]\n F --> G[\"Adaptive FPFH + match filter\"]\n G --> H[\"40× jittered RANSAC\"]\n H --> I[\"Otsu score + homology mask\"]\n I --> J[\"180° symmetry pick\"]\n J --> K[\"Point-to-plane ICP\"]\n K --> L{\"ICP improved score?\"}\n L -->|yes| M[\"Keep ICP pose\"]\n L -->|no| N[\"Keep pre-ICP pose\"]\n M --> O[\"Physical 4×4 + metadata\"]\n N --> O", "references": [ { "id": "porto-alpaca", "label": "ALPACA (Porto et al., 2021)", "blurb": "Original SlicerMorph ALPACA; Aurora keeps rigid stages only (no CPD landmark transfer).", "library": "Literature", "symbol": "Porto et al., 2021", "url": "https://doi.org/10.1111/2041-210X.13689" }, { "id": "rusu-fpfh", "label": "FPFH descriptors", "blurb": "Fast Point Feature Histograms used for correspondence before RANSAC.", "library": "Literature", "symbol": "Rusu et al., 2009", "url": "https://doi.org/10.1109/ROBOT.2009.5152473" }, { "id": "open3d-ransac-icp", "label": "Open3D RANSAC / ICP", "blurb": "Feature matching, RANSAC pose, and point-to-plane ICP refinement backends.", "library": "Open3D", "symbol": "open3d.pipelines.registration", "url": "https://www.open3d.org/docs/latest/tutorial/Advanced/global_registration.html" } ] }, "related": [ "align-to-reference", "alpaca-get-outer-mesh", "alpaca-create-landmarks-from-mesh", "normalize-ants-rigid-options", "invert-alignment-landmarks" ], "video": null, "images": [ { "caption": "C57 ALPACA debug clouds: misaligned → center/normalize → RANSAC → ICP → final rigid pose.", "path": "/static/docs-figures/docs_alpaca_align_landmarks_to_mesh.gif" } ], "outputs": [ { "name": "transformation_matrix", "type": "np.ndarray (4x4)", "description": "Rigid/similarity transform returned by the alignment (rotation + RMS-descaled residual translation; see algorithm notes)." }, { "name": "source_centroid", "type": "np.ndarray (3,)", "description": "Centroid of the source cloud used during normalization." }, { "name": "target_centroid", "type": "np.ndarray (3,)", "description": "Centroid of the target cloud used during normalization." }, { "name": "scale", "type": "float", "description": "Scale factor when scaling_mode is enabled; identity scale otherwise." }, { "name": "chosen_transform", "type": "np.ndarray (4x4)", "description": "Internal normalized-space transform selected by the pipeline before world remapping." }, { "name": "normalization_scale_factor", "type": "float", "description": "Shared RMS normalization scale applied to both clouds." }, { "name": "target metadata (side effect)", "type": "JSON file", "description": "When update_alignment_metadata is true, alignment success/parameters are written to target_metadata_path." } ] } ] }, { "id": "ants-rigid-options", "title": "ANTs rigid options", "methods": [ { "id": "normalize-ants-rigid-options", "title": "Normalize ANTs Rigid Options", "keywords": [ "ANTs", "rigid registration", "aff_metric", "iterations", "request parsing" ], "summary": "Parses and validates the ants_rigid_options object sent with Align to Reference when method is ants. Invalid or missing fields fall back to safe defaults and are clipped to allowed ranges so the UI can expose advanced ANTs tuning without risking unstable registration parameters.", "options": [ { "name": "request_data", "type": "dict", "default": "required", "description": "Full POST body; reads request_data['ants_rigid_options'] when present." }, { "name": "ants_rigid_options.aff_metric", "type": "str", "default": "mattes", "description": "ANTs affine metric; allowed: mattes, GC, meansquares." }, { "name": "ants_rigid_options.aff_iterations", "type": "list[int]", "default": "[1000, 800, 600, 100]", "description": "Per-level iteration counts; each value clipped to 0–5000." }, { "name": "ants_rigid_options.aff_shrink_factors", "type": "list[int]", "default": "[6, 4, 2, 1]", "description": "Multi-resolution shrink factors; each value clipped to 1–16." }, { "name": "ants_rigid_options.aff_smoothing_sigmas", "type": "list[int]", "default": "[3, 2, 1, 0]", "description": "Gaussian smoothing sigmas per level; each value clipped to 0–8." }, { "name": "ants_rigid_options.aff_sampling", "type": "int", "default": "48", "description": "Metric sampling percentage; clipped to 8–256." }, { "name": "ants_rigid_options.aff_random_sampling_rate", "type": "float", "default": "1.0", "description": "Random subsampling rate for metric evaluation; clipped to 0.01–1.0." }, { "name": "ants_rigid_options.grad_step", "type": "float", "default": "0.15", "description": "Gradient step size; clipped to 0.001–1.0." }, { "name": "ants_rigid_options.use_histogram_matching", "type": "bool", "default": "true", "description": "Enable ANTs histogram matching between fixed and moving images." }, { "name": "ants_rigid_options.singleprecision", "type": "bool", "default": "true", "description": "Run ANTs registration and apply_transforms in single precision." } ], "algorithm": { "text": "Extract ants_rigid_options from the request dict (or use empty dict). Validate aff_metric against an allow-list. Clip each numeric field to documented min/max bounds; replace invalid tuples with hard-coded defaults. Return a normalized dict consumed directly by ants.registration and ants.apply_transforms in AlignToReferenceView.", "math": "", "diagram": "flowchart TD\n A[\"request_data\"] --> B{\"ants_rigid_options dict?\"}\n B -->|\"no\"| C[\"Use empty dict\"]\n B -->|\"yes\"| D[\"Read each field\"]\n C --> E[\"Clip and default\"]\n D --> E\n E --> F[\"Return normalized opts\"]" }, "related": [ "align-to-reference", "registration-tools-min-max-normalize", "registration-tools-restore-range" ], "video": null } ] }, { "id": "voxel-elastic-registration", "title": "Voxel Elastic Registration", "methods": [ { "id": "elastic-registration", "title": "Elastic registration (voxel / ANTs SyN)", "keywords": [ "elastic", "SyN", "ANTs", "non-rigid", "deformation field", "voxel", "registration" ], "summary": "Voxel half of Aurora’s Elastic registration UI (POST /elastic-registration/). After rigid alignment has placed subjects on the reference grid, this endpoint runs ANTs SyNOnly non-linear registration so each moving NIfTI deforms onto the reference intensity/geometry. Unlike mesh NR-ICP, it stores forward/inverse deformation fields and reports Dice, NCC, and intensity error in addition to the shared surface score.\n\nUse this for volumetric CT/MRI subjects. Preserved PLY subjects are excluded here and handled by MeshElasticRegistrationView. Mixed projects expose both tools under the same Elastic registration menu.", "options": [ { "name": "directory", "type": "string", "default": "required", "description": "Project root." }, { "name": "reference", "type": "string", "default": "required", "description": "Fixed image stem or atlas. Preserved-mesh references are temporarily voxelized for movers." }, { "name": "quick_registration", "type": "string", "default": "UI guided presets or custom", "description": "Preset key such as guided-{mattes|cc|demons}-{1-5}, legacy low-fast/mid/high, or custom." }, { "name": "affine_initialization", "type": "boolean", "default": "false", "description": "Optional affine stage whose displacement initializes SyN." }, { "name": "double_pass", "type": "boolean", "default": "false", "description": "Second SyN pass with refined parameters." }, { "name": "use_mask / prefer_mask / reference_mask_only", "type": "boolean", "default": "UI toggles; reference_mask_only default true", "description": "Restrict registration to mask support. use_mask is mutually exclusive with do_dice_adjustment in the UI." }, { "name": "do_dice_adjustment", "type": "boolean", "default": "false", "description": "Post-hoc threshold search (±1500) to improve Dice on overlapping foreground." }, { "name": "use_distance_map", "type": "boolean", "default": "false", "description": "Register Euclidean distance transforms instead of raw intensities." }, { "name": "flagFilter", "type": "string", "default": "off", "description": "off | exclude | only." }, { "name": "elastic_custom_options", "type": "object", "default": "see normalize_elastic_custom_options", "description": "Custom SyN knobs: reg_iterations, syn_metric (CC|mattes|demons|meansquares), syn_sampling, grad_step, flow_sigma, total_sigma, singleprecision." } ], "outputs": [ { "name": "{scan}_edit_{n}_elastic.nii.gz", "type": "file", "description": "Warped moving volume in reference space." }, { "name": "{scan}_edit_{n}_elastic_fwd.nii.gz / _inv.nii.gz", "type": "file", "description": "Forward and inverse deformation fields (plus .mat if an affine component exists)." }, { "name": "lossy elastic preview", "type": "file", "description": "Downsampled preview when lossy_compression is configured." }, { "name": "elastic_to / elastic_dice / elastic_ncc / elastic_surface_distance / elastic_error", "type": "JSON fields", "description": "Reference name; Dice; normalized cross-correlation; shared surface-within-τ fraction; intensity MAE in overlap." } ], "algorithm": { "text": "Goal: non-linearly warp each voxel subject onto the reference volume after rigid alignment has matched grid shape and pose.\n\n0) Load the fixed image (latest reference NIfTI, or build_temporary_mesh_reference_volume when the reference is a preserved mesh). Enumerate voxel movers; exclude reference, faulty scans, subjects already ending in *_elastic.nii.gz, and apply flagFilter. Movers whose array shape ≠ reference grid fail with 400 — re-run Align to reference first.\n\n1) Per subject prep: optionally downsample if any axis > 512³ for the SyN solve; min_max_normalize to [−1,1]; optional EDT distance-map mode; optional masks (use_mask / prefer_mask / reference_mask_only).\n\n2) Optional affine_initialization → displacement field used as SyN initial transform.\n\n3) ants.registration with type_of_transform SyNOnly using the selected guided preset or elastic_custom_options (metric, iterations, sampling, sigmas, grad_step). Optional double_pass runs a second SyN with refined settings. If multiple .nii.gz transforms are produced, compose them into a single warp field.\n\n4) ants.apply_transforms on the original-resolution moving image (and paired mask with nearestNeighbor). Optional do_dice_adjustment searches thresholds around the current metadata threshold to improve Dice.\n\n5) Metrics & save: Dice on thresholded foreground; surface score with τ=6×voxel_size (same formula as mesh elastic); NCC on the union mask; intensity MAE as elastic_error. Write *_elastic.nii.gz, fwd/inv fields, lossy preview, update elastic_* metadata, stream progress. Batch continues on per-scan errors.", "math": [ { "equation": "\\mathrm{Dice} = \\frac{2\\,|A \\cap B|}{|A| + |B|}", "caption": "Overlap of thresholded registered vs reference foregrounds; optional ternary search of the subject threshold in a ±1500 intensity window maximizes Dice." }, { "equation": "\\mathrm{score} = 1 - \\frac{1}{N}\\sum_i \\mathbb{I}(d_i > 6\\Delta)", "caption": "Fraction of outer-shell vertices whose bidirectional surface distance stays within six voxels; Δ = voxel_size; fixed multiplier 6 (paper §2.5 / _mesh_elastic_error_threshold_mm)." }, { "equation": "\\mathrm{NCC} = \\mathrm{corr}(I, J)\\big|_{\\mathrm{union\\ mask}}", "caption": "Normalized cross-correlation of intensities on the union of thresholded reference and registered-subject masks; reported on [−1, 1]." } ], "diagram": "flowchart TD\n A[\"POST /elastic-registration/\"] --> B[\"Load fixed volume / temp mesh voxelization\"]\n B --> C[\"Filter voxel movers; require matching grid shape\"]\n C --> D[\"For each subject\"]\n D --> E[\"Optional downsample; normalize; optional EDT / masks\"]\n E --> F{\"affine_initialization?\"}\n F -->|\"yes\"| G[\"Affine → displacement init\"]\n F -->|\"no\"| H[\"SyNOnly registration\"]\n G --> H\n H --> I{\"double_pass?\"}\n I -->|\"yes\"| J[\"Second SyN refine\"]\n I -->|\"no\"| K[\"Compose warp fields if needed\"]\n J --> K\n K --> L[\"apply_transforms at full resolution\"]\n L --> M{\"do_dice_adjustment?\"}\n M -->|\"yes\"| N[\"Threshold search for Dice\"]\n M -->|\"no\"| O[\"Dice, surface, NCC, MAE metrics\"]\n N --> O\n O --> P[\"Save _elastic.nii.gz + fwd/inv fields + metadata\"]\n P --> D\n D --> Q[\"200 batch complete\"]" }, "related": [ "align-to-reference", "mesh-elastic-registration", "registration-tools-overview", "registration-tools-min-max-normalize", "registration-tools-calculate-affine-field" ], "video": null, "images": [ { "caption": "Voxel anatomy warps from pre-elastic to the SyN output; checkerboard confirms fit to the reference volume.", "path": "/static/docs-figures/docs_elastic_registration.gif" } ] } ] }, { "id": "landmark-inversion-export", "title": "Landmark Inversion & Export", "methods": [ { "id": "invert-alignment-landmarks", "title": "Invert Alignment Landmarks", "keywords": [ "landmarks", "inverse transform", "pre-alignment", "morphometrics", "multi-bone alignment" ], "summary": "Inverts landmarks placed on rigidly aligned scans back into a shared pre-alignment coordinate space (physical mm at reference voxel spacing). Use this after rigid alignment when you need landmark coordinates comparable across subjects or across multiple bones aligned separately from the same scan — for example before statistical shape analysis or when exporting homologous points that were placed on aligned volumes.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Project root directory." }, { "name": "scan_names", "type": "list[str]", "default": "null", "description": "Optional subset of subject names to process; all extracted subjects when omitted." }, { "name": "output_suffix", "type": "str", "default": "pre_alignment", "description": "Suffix for output filenames: {scan_name}_{output_suffix}_landmarks.json." } ], "algorithm": { "text": "1) Resolve subject list (explicit scan_names or all extracted folders), skipping faulty scans unless they are the reference. 2) For the reference subject, copy the latest edit landmarks unchanged. 3) For aligned subjects, load alignment_calculated_parameters from metadata and pick the latest landmark file from edits before any elastic edit. 4) Apply the inverse rigid pipeline: canvas mm → reference voxels, inverse rotation about the stored rotation center, subtract padding, undo scale_match, convert back to mm. 5) Write {scan}_{output_suffix}_landmarks.json and return per-subject status.", "math": [ { "equation": "v_{\\mathrm{pad}} = R^\\top (v_{\\mathrm{c}} - c_{\\mathrm{c}}) + c_{\\mathrm{rot}}", "caption": "Undo canvas rotation: v_c canvas voxel; c_c canvas centroid; c_rot original rotation center." }, { "equation": "v_{\\mathrm{pre}} = (v_{\\mathrm{pad}} - p_{\\mathrm{low}}) / s", "caption": "Remove padding_low and undo scale_match s." }, { "equation": "x_{\\mathrm{mm}} = v_{\\mathrm{pre}} \\cdot \\delta_{\\mathrm{ref}}", "caption": "Convert pre-scale voxels to millimeters with reference voxel size." } ], "diagram": "flowchart TD\n A[\"POST directory\"] --> B[\"Resolve subjects\"]\n B --> C{\"reference subject?\"}\n C -->|\"yes\"| D[\"Copy latest edit landmarks\"]\n C -->|\"no\"| E[\"Load alignment_calculated_parameters\"]\n E --> F[\"Pick pre-elastic landmark edit\"]\n F --> G[\"Inverse rotation padding scale\"]\n D --> H[\"Write pre_alignment landmarks JSON\"]\n G --> H" }, "related": [ "align-to-reference", "export-inverted-landmarks" ], "video": null, "images": [ { "caption": "Red landmarks on the aligned mesh fly under Aurora’s inverse rigid (Rᵀ·T), then settle on the raw pre-alignment mesh.", "path": "/static/docs-figures/docs_invert_alignment_landmarks.gif" } ] }, { "id": "export-inverted-landmarks", "title": "Export Inverted Landmarks", "keywords": [ "CSV export", "landmarks", "pre-alignment", "morphometrics", "spreadsheet" ], "summary": "Collects inverted pre-alignment landmark files from every non-faulty subject and returns them as a downloadable CSV spreadsheet. Run this after Invert Alignment Landmarks when you need landmark coordinates in R, Python, or external morphometrics tools — each row is one subject with landmark_x/y/z columns.", "options": [ { "name": "directory", "type": "str", "default": "required", "description": "Project root directory containing extracted/." }, { "name": "output_suffix", "type": "str", "default": "pre_alignment", "description": "Suffix matching inverted landmark files: {scan_name}_{output_suffix}_landmarks.json." } ], "algorithm": { "text": "1) List all subject folders under extracted/, skipping faulty scans. 2) For each subject, load {scan}_{output_suffix}_landmarks.json if present. 3) Flatten landmark positions into columns (landmark_i_x/y/z; semi_ prefix for semi-automatic landmarks). 4) Build a pandas DataFrame sorted by subject name, write a timestamped CSV to .temp/, and stream it as a FileResponse attachment.", "math": "", "diagram": "flowchart TD\n A[\"POST directory + output_suffix\"] --> B[\"Scan extracted subjects\"]\n B --> C[\"Load inverted landmark JSON\"]\n C --> D[\"Flatten to x y z columns\"]\n D --> E[\"Build DataFrame\"]\n E --> F[\"Write CSV and download\"]" }, "related": [ "invert-alignment-landmarks" ], "video": null } ] }, { "id": "registration-tools", "title": "Registration Tools", "methods": [ { "id": "registration-tools-overview", "title": "RegistrationTools", "keywords": [ "registration utilities", "ANTs", "normalization", "lossy compression", "landmarks", "background" ], "summary": "Shared helper class used across rigid alignment, elastic registration, rotation/crop views, preprocessing, and mask pipelines. It centralizes intensity normalization for ANTs, background estimation, lossy NIfTI preview generation, affine-to-displacement-field conversion for elastic initialization, and landmark coordinate propagation through geometric edits.", "options": [], "algorithm": { "text": "RegistrationTools is a stateless utility class instantiated where needed (e.g. AlignToReferenceView.registration_tools). Methods are composed in registration workflows: estimate background → normalize to [-1,1] for ANTs → register → restore original intensity range → save lossy previews and transform landmark sidecars.", "math": "", "diagram": "flowchart TD\n A[\"Volume or landmarks\"] --> B[\"Background estimation\"]\n B --> C[\"min_max_normalize\"]\n C --> D[\"ANTs registration\"]\n D --> E[\"restore_range\"]\n E --> F[\"save_as_lossy_nifti + landmark transforms\"]" }, "related": [ "align-to-reference", "registration-tools-calculate-affine-field", "registration-tools-save-lossy-nifti" ], "video": null }, { "id": "registration-tools-get-background-value", "title": "Get Background Value", "keywords": [ "background", "histogram", "intensity", "border pixels", "segmentation threshold" ], "summary": "Estimates the tissue-background intensity level from a 3D volume. Used before padding, resampling, and ANTs normalization so empty regions are filled with a statistically consistent background rather than zero or an arbitrary constant.", "options": [ { "name": "image_data", "type": "np.ndarray", "default": "required", "description": "3D volume array." }, { "name": "mode", "type": "str", "default": "peak-no-zero", "description": "Estimation strategy: peak-no-zero (Gaussian-smoothed histogram peaks) or border (2-voxel shell histogram)." }, { "name": "threshold", "type": "number", "default": "null", "description": "Optional intensity cutoff for border mode; voxels above threshold are excluded from the border shell." } ], "algorithm": { "text": "peak-no-zero: Gaussian smooth (sigma=3), find histogram peaks, pick the lowest-intensity among the two highest-frequency peaks, return the mean intensity in that bin. border: Gaussian smooth (sigma=1), collect 2-voxel-thick border voxels (optionally masked by threshold), return histogram mode or mean when variance is low.", "math": "", "diagram": "flowchart TD\n A[\"image_data\"] --> B{\"mode\"}\n B -->|\"peak-no-zero\"| C[\"Smooth histogram peaks\"]\n B -->|\"border\"| D[\"Sample 2px border shell\"]\n C --> E[\"Pick background intensity\"]\n D --> E" }, "related": [ "registration-tools-overview", "registration-tools-get-adjusted-background-value" ], "video": null }, { "id": "registration-tools-get-adjusted-background-value", "title": "Get Adjusted Background Value", "keywords": [ "background", "normalization", "linear mapping", "ANTs" ], "summary": "Maps a background intensity from the original value range into a new normalized range (typically [-1, 1] for ANTs). Use alongside min_max_normalize so padded and transformed regions keep the correct background level after intensity rescaling.", "options": [ { "name": "background_value", "type": "number", "default": "required", "description": "Background intensity in the original image range." }, { "name": "image_data", "type": "np.ndarray", "default": "required", "description": "Source volume used to determine original min/max." }, { "name": "new_min", "type": "number", "default": "required", "description": "Target minimum of the normalized range." }, { "name": "new_max", "type": "number", "default": "required", "description": "Target maximum of the normalized range." } ], "algorithm": { "text": "Linearly map background_value from [min(image_data), max(image_data)] to [new_min, new_max] using the same scaling as min_max_normalize.", "math": [ { "equation": "b' = \\frac{b - m}{M - m}(M' - m') + m'", "caption": "Map background b from [m, M] into the normalized display range [m', M']." } ], "diagram": "" }, "related": [ "registration-tools-min-max-normalize", "registration-tools-restore-range" ], "video": null }, { "id": "registration-tools-min-max-normalize", "title": "Min–Max Normalize", "keywords": [ "normalization", "intensity", "ANTs", "memory efficient" ], "summary": "Linearly rescales voxel intensities to a target range while preserving relative contrast. This is the standard preprocessing step before ANTs rigid and elastic registration in Aurora, which expect numerically stable floating-point images (commonly [-1, 1]).", "options": [ { "name": "image", "type": "np.ndarray", "default": "required", "description": "Input volume." }, { "name": "new_min", "type": "number", "default": "0", "description": "Target minimum after normalization." }, { "name": "new_max", "type": "number", "default": "1", "description": "Target maximum after normalization." } ], "algorithm": { "text": "Compute image min/max; if equal, return a constant array at new_min. Otherwise copy to float32, subtract min, divide by range, scale to [new_min, new_max], and clip in place.", "math": [ { "equation": "I' = \\frac{I - m}{M - m}(M' - m') + m'", "caption": "Linear min–max remap of image I from [m, M] onto [m', M']." } ], "diagram": "" }, "related": [ "registration-tools-restore-range", "normalize-ants-rigid-options" ], "video": null }, { "id": "registration-tools-restore-range", "title": "Restore Range", "keywords": [ "denormalization", "intensity", "ANTs output", "original dtype" ], "summary": "Inverse of min_max_normalize: maps a normalized registered volume back to the subject's original intensity range. Applied after ANTs apply_transforms so saved NIfTI edits retain familiar HU or grayscale values for thresholding and visualization.", "options": [ { "name": "image", "type": "np.ndarray", "default": "required", "description": "Normalized volume to restore." }, { "name": "original_min", "type": "number", "default": "required", "description": "Original intensity minimum to recover." }, { "name": "original_max", "type": "number", "default": "required", "description": "Original intensity maximum to recover." } ], "algorithm": { "text": "Normalize the current array to [0, 1], scale to [original_min, original_max], and clip in place — mirroring min_max_normalize in reverse.", "math": [ { "equation": "I_{\\mathrm{restored}} = \\frac{I - m}{M - m}(M_0 - m_0) + m_0", "caption": "Undo a temporary normalization using the original intensity bounds [m_0, M_0]." } ], "diagram": "" }, "related": [ "registration-tools-min-max-normalize", "align-to-reference" ], "video": null }, { "id": "registration-tools-save-lossy-nifti", "title": "Save as Lossy NIfTI", "keywords": [ "lossy compression", "JPEG", "preview", "downsample", "metadata" ], "summary": "Creates a downsampled, JPEG-compressed 8-bit NIfTI preview plus an isometric projection PNG, and appends scale metadata to the subject JSON. Used after alignment, elastic registration, rotation, crop, and preprocessing so the web viewer can load fast previews without storing full-resolution lossless copies.", "options": [ { "name": "image_data", "type": "np.ndarray", "default": "required", "description": "Full-resolution 3D volume to compress." }, { "name": "voxel_size", "type": "number", "default": "required", "description": "Voxel spacing in mm for the output affine." }, { "name": "json_file", "type": "str", "default": "required", "description": "Path to subject metadata JSON; lossy_compression entry is appended." }, { "name": "output_file", "type": "str", "default": "required", "description": "Destination path for the lossy .nii.gz file." } ], "algorithm": { "text": "1) Read resolution_factor from json lossy_compression metadata (default 2). 2) Mean-pool voxels with block_reduce after edge padding. 3) Linearly map pooled data to 0–255, JPEG-compress each axial slice (quality 70), preserving exact zeros. 4) Generate isometric projection PNG(s) via VTK raycasting with weighted-sum fallback. 5) Save NIfTI and append scale_factor, zero_point_shift, resolution_factor to lossy_compression in JSON.", "math": "", "diagram": "flowchart TD\n A[\"Full-res volume\"] --> B[\"Mean-pool by resolution_factor\"]\n B --> C[\"Map to 8-bit\"]\n C --> D[\"JPEG per slice\"]\n C --> E[\"Isometric projection PNG\"]\n D --> F[\"Save lossy NIfTI\"]\n F --> G[\"Update JSON metadata\"]" }, "related": [ "registration-tools-overview", "align-to-reference" ], "video": null }, { "id": "registration-tools-calculate-affine-field", "title": "Calculate Affine Transform as Field", "keywords": [ "affine registration", "displacement field", "ANTs", "SyN", "elastic initialization" ], "summary": "Converts an ANTs Affine transform into a dense displacement field on the fixed grid so it can initialize SyNOnly (which accepts fields, not affine matrices, as initial_transform).", "options": [ { "name": "fixed_image_data", "type": "ants.ANTsImage", "default": "required", "description": "Fixed (reference) ANTs image." }, { "name": "moving_image_data", "type": "ants.ANTsImage", "default": "required", "description": "Moving ANTs image to align to fixed." } ], "algorithm": { "text": "1) Run ants.registration Affine (Global Correlation) between fixed and moving. 2) Recover A ∈ R^{3×3} and translation b from the ITK .mat; re-center about rotation center c as t = b + c − A c. 3) For each voxel index i, physical p = D(i ⊙ s) + o; store u(i) = (A p + t) − p. 4) Copy u into a properly headered vector NIfTI (placeholder SyN with reg_iterations = [1,0,0,0] then overwrite voxels).", "math": [ { "equation": "\\mathbf{t} = \\mathbf{b} + \\mathbf{c} - A\\mathbf{c}", "caption": "Re-center ITK translation b about rotation center c for the linear part A." }, { "equation": "\\mathbf{u}(\\mathbf{i}) = (A\\mathbf{p} + \\mathbf{t}) - \\mathbf{p}", "caption": "Per-voxel displacement on the fixed grid; p = D(i ⊙ s) + o from direction, spacing, and origin." } ], "diagram": "flowchart TD\n A[\"Fixed + moving ANTs images\"] --> B[\"Affine registration\"]\n B --> C[\"Build physical voxel grid\"]\n C --> D[\"Apply affine per voxel\"]\n D --> E[\"Subtract to displacement field\"]\n E --> F[\"Write field NIfTI + return paths\"]" }, "related": [ "registration-tools-overview", "registration-tools-min-max-normalize" ], "video": null }, { "id": "registration-tools-load-nifti-mask-data", "title": "Load NIfTI Mask Data", "keywords": [ "mask", "NIfTI", "nii.mask.gz", "elastic registration" ], "summary": "Loads mask voxel data from a NIfTI path for elastic registration, including a temp-file workaround for Aurora's non-standard *.nii.mask.gz double suffix that NiBabel often cannot sniff. Used when reference and moving masks constrain the registration metric.", "options": [ { "name": "mask_path", "type": "str", "default": "required", "description": "Filesystem path to the mask NIfTI (including *.nii.mask.gz)." } ], "algorithm": { "text": "1) Try nib.load(mask_path).get_fdata(). 2) On ImageFileError, create a temporary file with suffix _mask_temp.nii.gz, copy the mask into it, load get_fdata from the temp path, then delete the temp file. 3) Return the voxel array to the caller.", "math": [], "diagram": "flowchart TD\n A[\"mask_path\"] --> B[\"nib.load.get_fdata\"]\n B -->|\"ok\"| C[\"Return array\"]\n B -->|\"ImageFileError\"| D[\"Copy to *_mask_temp.nii.gz\"]\n D --> E[\"Load temp + delete\"]\n E --> C", "references": [] }, "related": [ "registration-tools-overview" ], "video": null, "outputs": [ { "name": "(return)", "type": "ndarray", "description": "Floating-point voxel array from nib.load(...).get_fdata()." } ] }, { "id": "registration-tools-transform-landmarks-by-rotation", "title": "Transform Landmarks by Rotation", "keywords": [ "landmarks", "rotation", "crop", "coordinate propagation" ], "summary": "Propagates landmark coordinates through the same rotation, embedding offset, and final crop applied to a volume during manual rotation edits. Keeps landmark sidecars synchronized with geometric edits so downstream alignment and morphometrics stay consistent.", "options": [ { "name": "landmarks_list", "type": "list", "default": "required", "description": "Landmarks as [x,y,z] mm arrays or dicts with position and landmark_type." }, { "name": "voxel_size", "type": "number", "default": "required", "description": "Voxel spacing in mm." }, { "name": "rotation_matrix", "type": "np.ndarray (3,3)", "default": "required", "description": "Rotation matrix applied to the volume." }, { "name": "center_box", "type": "array-like (3,)", "default": "required", "description": "Rotation center in voxel space." }, { "name": "origin_offset", "type": "array-like (3,)", "default": "required", "description": "Embedding offset subtracted before rotation." }, { "name": "final_bbox_min", "type": "array-like (3,)", "default": "required", "description": "Minimum crop bounds in voxel space." }, { "name": "final_bbox_max", "type": "array-like (3,)", "default": "required", "description": "Maximum crop bounds in voxel space." }, { "name": "embedding_offset", "type": "array-like (3,)", "default": "required", "description": "Offset converting landmarks into the target embedding frame." } ], "algorithm": { "text": "For each landmark: convert mm → voxels, subtract embedding_offset, rotate about origin_offset, test inclusion in [final_bbox_min, final_bbox_max), subtract crop origin, convert back to mm. Landmarks outside the crop are excluded; survivors retain landmark_type.", "math": [ { "equation": "v' = R(v - o)", "caption": "v: landmark; o: origin_offset; R: rotation; keep v' inside the final bounding box." } ], "diagram": "flowchart TD\n A[\"Landmarks in mm\"] --> B[\"To voxel space\"]\n B --> C[\"Apply embedding offset\"]\n C --> D[\"Rotate\"]\n D --> E{\"Inside crop?\"}\n E -->|\"yes\"| F[\"Subtract crop origin to mm\"]\n E -->|\"no\"| G[\"Exclude landmark\"]" }, "related": [ "registration-tools-transform-landmarks-by-crop", "invert-alignment-landmarks" ], "video": null }, { "id": "registration-tools-transform-landmarks-by-crop", "title": "Transform Landmarks by Crop", "keywords": [ "landmarks", "crop", "padding", "coordinate propagation" ], "summary": "Updates landmark positions after a volumetric crop and padding operation. Used when subjects are cropped to a region of interest so landmark JSON sidecars remain aligned with the edited volume geometry.", "options": [ { "name": "landmarks_list", "type": "list", "default": "required", "description": "Landmarks as [x,y,z] mm arrays or dicts with position and landmark_type." }, { "name": "voxel_size", "type": "number", "default": "required", "description": "Voxel spacing in mm." }, { "name": "z_min", "type": "number", "default": "required", "description": "Crop minimum along axis 0 (voxel indices)." }, { "name": "z_max", "type": "number", "default": "required", "description": "Crop maximum along axis 0 (voxel indices)." }, { "name": "y_min", "type": "number", "default": "required", "description": "Crop minimum along axis 1 (voxel indices)." }, { "name": "y_max", "type": "number", "default": "required", "description": "Crop maximum along axis 1 (voxel indices)." }, { "name": "x_min", "type": "number", "default": "required", "description": "Crop minimum along axis 2 (voxel indices)." }, { "name": "x_max", "type": "number", "default": "required", "description": "Crop maximum along axis 2 (voxel indices)." }, { "name": "padding", "type": "number", "default": "required", "description": "Padding added in voxels after cropping." } ], "algorithm": { "text": "Convert each landmark from mm to voxels; if inside the crop box [z_min,z_max) × [y_min,y_max) × [x_min,x_max), subtract crop origin, add padding offset, convert back to mm and retain landmark_type; otherwise exclude the landmark.", "math": [ { "equation": "v_{\\mathrm{crop}} = v - (z_0, y_0, x_0) + p", "caption": "Shift landmarks into the cropped frame; p: padding along each axis." } ], "diagram": "flowchart TD\n A[\"Landmarks in mm\"] --> B[\"To voxels\"]\n B --> C{\"Inside crop ROI?\"}\n C -->|\"yes\"| D[\"Subtract crop origin add padding\"]\n D --> E[\"Back to mm\"]\n C -->|\"no\"| F[\"Exclude\"]" }, "related": [ "registration-tools-transform-landmarks-by-rotation" ], "video": null } ] } ] } ] }