How stacking works

This page explains what Helia does between a capture and a finished image, in plain terms, and which setting changes which step. The precise maths and the module layout are in the Architecture page.

The problem

Between the Sun and your telescope is a few kilometres of moving air. In any one short exposure the image is shifted, stretched and blurred by it, and the pattern changes within milliseconds. Most frames are poor. Every now and then, for a moment, the air over your telescope is steady and a frame is sharp.

Lucky imaging records thousands of frames, keeps the lucky ones, lines them up and averages them. Averaging many aligned frames also cancels random noise.

0. Calibration (optional)

Before anything else, frames can be calibrated: a master dark is subtracted, the result is divided by a master flat and bad pixels are replaced by the median of their neighbours of the same colour. It happens as each frame is read, on the stored pixels before any demosaicing, so the ranking, the alignment and the stack all see calibrated frames. The reason it comes first is that the sensor's pattern does not move with the Sun: alignment warps the picture, and a pattern that stayed put turns into detail that moves. See Calibration.

1. Ranking the frames

Helia gives every frame a sharpness score. The default metric, gradient energy, is the average squared difference between neighbouring pixels: a sharp frame has strong edges and fine contrast, a blurred one does not. The alternative, Laplacian variance, measures the spread of the second derivative. Choose between them with --metric or the sharpness metric in the app.

Two details make the score trustworthy on the Sun:

• A small amount of image noise would raise a gradient score, so Helia estimates the noise floor of each frame and subtracts it.
• Overall brightness and the pedestal of the sky do not change the score, so a frame that is merely brighter does not rank higher.

The frames are then sorted best first. You choose how many to keep with --keep (a share such as 25%, the default; 12% for white light and Na D) or --keep-frames (a count). The app draws the sorted scores as the Quality, sorted curve, with a dashed line where the kept frames end.

The frames you keep are not all treated alike. Each gets a weight between 0.2 (the worst kept frame) and 1 (the best), following a smooth S-shaped curve, so sharp frames dominate the average but borderline frames still help with the noise. This is a smooth curve and not a cut-off.

2. A reference to line up against

To align frames you need something to align them to. Helia averages the best few frames (five by default), after aligning them to each other, into a reference image. Because they are the sharpest frames, the reference is already a decent picture, and the other frames are measured against it.

3. Aligning the whole frame

The air tilts the image, so every frame sits a little differently on the sensor. Helia finds each frame's shift relative to the reference by one of two methods. This is the global alignment. It removes the largest part of the blur, which is plain image motion.

• Phase correlation compares the frames in the frequency domain, which is fast and robust, and then refines the answer to a fraction of a pixel. It needs detail in the picture to lock onto, and it is the method for a cropped region of the Sun. With a GPU it runs on the device using Helia's own Fourier transform; the CPU fits the peak either way. See GPU acceleration.
• Limb fit finds the edge of the solar disc, fits a circle to it, and takes the shift from where the circle's centre moved. A full-disc frame has no small patch to correlate, but it has a sharp circular edge. Helia casts 720 rays out from the disc, finds the fall in brightness along each to a fraction of a pixel, and fits the circle with a robust fit that ignores prominences, flares and sunspot edges. It runs on the CPU.

Helia chooses for you. With --align auto (the default) it fits the limb of the reference. If the whole disc is in the frame and the fit is confident (0.8 or more, see below) it aligns by limb fit. If the frame cuts the limb off, there is no disc, or the fit is shaky, it uses phase correlation. Force one with --align phase or --align limb. The choice, the reason and the confidence are shown at the end of a run on the command line (the align: line), in the app (under the Stack button, with the reason on hover) and by the API (summary.alignment).

Confidence is a number from 0 to 1 for each frame.

• For the limb fit it is high when the edge is located precisely and consistently, and the visible limb is spread widely enough to pin the centre down. It comes from the predicted error of the centre in pixels (from the scatter of the edge points, and from refitting without each eighth of the limb in turn): 0.5 is a predicted error of 0.15 pixel and 0.9 is about 0.05 pixel. It is reduced when some edge points disagree with the circle or many rays found no edge.
• For phase correlation it comes from the sharpness of the correlation peak against its surroundings: 0 for a peak-to-sidelobe ratio of 4 or less, 1 at 10 or more.

A frame whose limb fit is not confident (below 0.5) is aligned by phase correlation instead, and the summary counts those frames. When phase correlation runs on the GPU, which does not return a confidence, Helia re-measures eight frames spread over the candidates on the CPU and reports those, marked sampled.

When the limb is cut by the frame the radius is held at the reference's and only the centre is fitted, because a short arc cannot tell a bigger circle from a shifted one. That is why auto leaves such captures to phase correlation.

How accurate is it?

Measured, not promised. On synthetic discs with known shifts (the tests in crates/helia-core/src/align/limb.rs and tests/limb_align.rs) the shift error of the limb fit is:

On real frames there is no ground truth, so the limb fit is compared with phase correlation on the same frames (the best 25% of the first 200 frames of each capture, aligned to the pipeline's reference, CPU run on a 24-thread server):

So the limb fit and phase correlation agree to about a tenth of a pixel when the limb is clean, and auto uses the limb only then. In the other captures the seeing deforms the limb enough that its centre wanders by several tenths of a pixel against the content, the confidence is lower, and auto stays with phase correlation. In a global-only stack of these captures the two methods gave the same sharpness to within 0.5% (limb over phase correlation: 1.000 for 15_32_09.avi, 1.002 for 15_20_57.ser, 1.005 for Sun_Test.ser, 1.000 for 14_42_57.avi), which is a tie, not a win. Whether a better result is possible with the limb on the noisier captures is not established; use --align limb to try it.

4. Aligning small patches: alignment points

Global alignment is not enough on the Sun. The air does not bend the image by the same amount everywhere: over a patch a few arcseconds across the distortion is roughly uniform, but the disc covers tens of thousands or more of such patches. After a global shift, one part of the disc may still be a pixel off while another is spot on, and the sharp moments differ from place to place too. So Helia aligns many small points as well as the whole frame.

Helia lays a grid of overlapping square windows, the alignment points, over the reference. The default is 64 pixels across, overlapping by half; you change the size with --ap-size or the point size in the app.

For each point and each frame Helia measures the local shift by correlating the window against the reference, searching a small distance around the global shift, and scores how sharp that window is in that frame. Then each point picks its own best frames: a frame that was sharp over the left of the disc and soft on the right contributes to the left only. To make this possible, Helia aligns twice as many frames as you keep, so each point has a choice.

Points are placed only where there is something to align: the centre of a point's window must be inside the solar disc (see Keeping alignment off text and sky). Some points cannot be measured: flat regions, flat regions, or a straight stretch of limb, where the pattern fixes the shift across the edge but not along it. Those points are not allowed to guess. They take the distance-weighted average of their neighbours' shifts instead (or the global shift when there are no neighbours). The app reports how many points are active, meaning bright and detailed enough to be measured.

Alignment points are correlated on a smoothed copy of the frame with the slow brightness gradient removed, because on low-contrast detail such as H-alpha the limb darkening would otherwise flatten the correlation peak.

Choosing the points automatically

--ap-auto (the Auto chip) chooses the point size, spacing and search range from the capture instead of one size for everything.

• Scale. The reference's limb fit gives the radius in pixels; the Sun is 959.6 arcseconds in radius, so the scale is that over the radius. The window is the size of the detail in arcseconds over the scale: 3.75 arcseconds (2.5 granules) for white light and sodium, 90 arcseconds for the other filters (a starting point; no chromospheric capture has tuned it), rounded to a multiple of 8 and kept within 32 to 96 pixels. Without a limb the recommended size for the frame is used.
• Seeing. A dozen of the frames about to be stacked are registered globally and a coarse grid of 48 pixel points is measured in each. The local warp is the root mean square distance of a point's shift from the median shift of its frame, over the locked points of all the probe frames. The search radius covers three times that warp, between 4 and 12 pixels.
• Spacing is half a window, as before.

What the real captures said. All four are ASI482MC sodium captures, stacked with the Na D preset on the CPU; the figure is the granulation band (1 to 4 px) of helia measure at the disc centre, so higher is more granulation contrast:

• At these scales a granule is under three pixels, so the rule lands on the 32 pixel floor on every capture. Helia's automatic size is the preset's, and the auto run's granulation figure is the --ap-size 32 one to the last digit of the table. The window only grows on a scale finer than about 0.12 arcsec per pixel, or for a filter with larger detail.
• Smaller windows read higher in the granulation band (at 24 px: 1.884% on Sun_Test.ser, 2.064% on 15_20_57.ser, 0.980% on 15_32_09.avi, 0.210% on 14_42_57.avi). On 14_42_57.avi, where only 63% of the limb is in view and the contrast is low, the meso band (4 to 12 px) rises from 0.39% at 32 px to 0.54% at 24 px and 1.38% at 12 px. That is blocking from windows that lock on noise, not detail, so 32 pixels stays the floor.
• Spacing, with auto's 32 pixel windows: a step of 8, 12 or 16 px (75% to 50% overlap) gave the same granulation band on the first three captures (within 1%), while a step of 24 or 32 px (little or no overlap) lost 1% to 5% of it and blocked 14_42_57.avi (meso 0.39% to 1.6%). Half a window is kept.
• The search radius made no measurable difference here (14_42_57.avi at 7 px against 4: 0.169% against 0.170%). It is there so a capture with large local motion does not lose its points at the edge of the search.

Refining the shifts

Two options, both off by default, work on the table of shifts the points produce (#4).

• --ap-passes 2 stacks once, then measures every point again against that stack instead of against the reference, which is only the best few frames averaged and so noisier and softer. The second table is what the final stack uses. The cost is one more measuring pass and one more stack: about 30% more time on the sodium captures (3.0 to 3.8 s on the GPU, 4.6 to 6.7 s on the CPU for Sun_Test.ser).
• --ap-smooth on replaces a shift that sits further from the median of its neighbours (the points within one and a half grid steps) than three times the neighbours' own robust spread, and further than 0.3 px. Such a point has usually locked on the next granule over. A real local distortion varies smoothly, so its neighbours spread out and nothing is replaced. --ap-smooth off is the default.

What the real captures said. The same four sodium captures, Na D preset, GPU. The three granulation columns are the bands of helia measure (fine 0 to 1 px, granulation 1 to 4 px, meso 4 to 12 px). The residual is what --ap-residual prints: the shifts are measured once more against the finished stack, and the figure is the RMS distance in pixels from the shifts that were used (smaller means the stack agrees better with the frames it was made from).

• The second pass cuts the residual by a factor of 1.7 to 3.6 on every capture, and adds 1.8% to the granulation band on Sun_Test.ser (1.847% to 1.880%) and 1.2% on 14_42_57.avi, the two captures with the larger local warp (0.88 and 1.59 px). Where the warp is small (0.23 px on 15_20_57.ser and 15_32_09.avi) the reference is already good and nothing moves.
• Smoothing changed the granulation band by less than 0.1% everywhere. It replaces 2.6% of the shifts on 14_42_57.avi, the low-contrast capture where mismatches are common, and lowers the residual there by 2%. Elsewhere it makes no measurable difference.
• Synthetic data with a known truth (48 frames of a warped, noisy synthetic sun, 32 px points, the test refinement_does_not_hurt_on_noisy_warped_data) did not show the second pass as an accuracy gain: the RMS error against the clean sun was 0.0471 with one pass and 0.0528 with two at noise 0.02, and 0.0556 against 0.0606 at noise 0.2, with sharpness slightly higher (1% at both). That warp is much stronger than the seeing in the captures (2 px amplitude over 90 px, against windows of 32 px), so a window averages a distorted patch whichever reference it uses. Treat the second pass as a refinement for captures with real local motion, and confirm with --ap-residual and helia measure.

Keeping alignment off text and sky

Capture programs such as SharpCap can draw the date and time into the picture. That text is bright, sharp and different in every frame, which makes it the worst possible thing to align on, and the empty sky around the disc has nothing to align on at all. Before it scores anything, Helia therefore:

• Finds burned-in text. It looks at about 16 frames spread over the capture for pixels that flip between white and dark, in a small area at a frame edge. The unchanging part of the same line (the date) is added to it. The area is then replaced by one flat level, the median of the pixels around it, in every frame, so that scoring, global alignment, alignment points and the stack all agree. A timestamp therefore disappears from the result instead of being smeared. Switch this off with --no-overlay-detect.
• Fits the limb. A circle is fitted to the steepest edge between sky and disc on the reference image. The fit ignores points that disagree with the circle (a prominence, a flare) and works when the frame cuts off part of the disc. An alignment point is used only when the centre of its window is inside the circle. --prominence-band <px> allows points that far outside the limb, so that they can follow prominences; the default is 0. If no clear disc is found (a capture of something other than the Sun) nothing is masked this way. Switch it off with --no-disc-mask.
• Honours your rectangles. --exclude x,y,w,h (repeat it for several), the exclude list of an API job, and the Exclude regions list in the app keep anything else out: a logo, a stray label, a hot pixel cluster. No alignment point window may overlap such a rectangle, and like detected text the area is flat in the result.

5. Stacking

Each point's chosen frames are moved by their measured shifts (sampled at sub-pixel positions) and combined as a weighted mean. That is the default, and for a seeing-selected stack it is the right answer: the weights already say how much each frame is worth.

Three optional guards deal with samples that do not belong, such as a hot pixel a dark frame missed, a satellite or plane trail, a passing bird or a cloud dimming a few frames. They are chosen with --robust (or robust in an API job, or the Outliers chips in the app) and all work the same way: a first pass over the frames finds the weighted mean and spread at every pixel, a second pass decides what each sample is worth from those. Memory does not grow with the number of frames, and the capture is read twice instead of once.

--sigma-clip K is the older spelling of --robust sigma --robust-k K. At a pixel with fewer than three samples nothing is changed. All of them are off by default, and none is a cure for bad alignment.

How they compare, measured on synthetic data with a known answer (200 frames of a 48 by 48 scene, noise 0.02 on a signal of 0.2 to 0.8, quality weights 0.5 to 1.5, default K; cargo test -p helia-core --all-features --test robust_stack -- --nocapture, commit f7aa530). Errors are RMSE against the truth, times 1000:

On clean data the cost is small: against the weighted mean, the mean level changes by at most 0.001% and the contrast of the fine texture by 0.04% (sigma), 0.13% (Huber) and 0.008% (winsor), well inside the 1% budget. A 16 by 16 patch that 5% of the frames sample 2.5 px away (a bad alignment point shift) has an RMSE of 1.86 for the mean and 1.70, 1.71 and 1.76 for sigma, Huber and winsor: a modest gain, not a repair.

Two things to know. The spread comes from the same samples, so a heavy contamination inflates it and the soft forms then let part of an outlier through; a hard clip is the best at removing gross outliers (a hot pixel, a trail) and the soft forms pay for it only in the 5% case. In exchange the soft forms are gentler on real, asymmetric spread: with one sample in five pulled down by 0.05 the result moves from the weighted mean by 0.89 (sigma), 1.29 (Huber) and 0.39 (winsor), times 1000. Winsor is therefore the gentlest on real data and sigma the strongest on gross outliers; Huber at its customary 1.345 sits between them on outliers and moves the result most on real spread, so raise K for it with --robust-k if that matters. With real lucky-imaging stacks, where the spread between frames is partly signal, leave it off unless the capture has an artefact to remove.

6. Blending and writing

The per-point stacks overlap, so Helia blends them with smooth raised-cosine weights that sum to one where windows overlap. This avoids visible seams. Where no point could be measured, the globally aligned stack fills in. The result is scaled so that its 99.99th percentile is 1 (full scale), and written as a 16-bit TIFF, an 8-bit PNG or a 32-bit FITS. See Output formats.

Cropping to the disc

--autocrop (the Crop chip) runs after the stack. The limb is fitted on the stacked image, which is sharper and quieter than a single frame, and the result is cut to a square around it with a margin, 10 per cent of the radius by default. The pixels are the stack's own (cutting the uncropped image by the same rectangle gives the same result), and the disc is reported in the cropped image's pixels, so a FITS header's limb and plate scale stay right. A disc near an edge moves the square inwards, a disc larger than the frame keeps the whole of that side, and with no clear disc nothing is cropped. On the four captures the crops were 979 x 979, 978 x 978, 490 x 490 and 1460 x 1080 pixels (the last disc is larger than the frame, so its short side is whole).

7. Colour (optional)

Sharpening and the everyday adjustments are applied after the stack is finished, as a recorded list of operations on the mono stack (see Sharpening and adjustments). The stack itself is never changed.

Colour is applied after the sharpening, to the finished image, and only when you ask for it. The stack is a single channel; a preset for the filter (orange for H-alpha, violet for calcium and so on) maps its grey values to colours through a table, for a 16-bit RGB TIFF or an 8-bit PNG. FITS stays mono. The colour is a presentation choice, and the file records it. See Colourising.

True colour is the other way round: with --colour true the three planes of a raw colour capture are stacked on the one alignment above (the frames are scored and aligned once, on the luminance, and the planes use the same shifts, alignment point field and weights), the red and blue stacks are registered to the green by phase correlation to remove dispersion, and a white balance and one common scale make the picture. See True colour.

What Helia does not do yet

• Two-pass refinement of the alignment points against a first stack (#4).
• Drizzle (#6).

See the Roadmap.