Noise Reduction or Resolution Quadrupling by adding more photos together

Shows previously invisible things

A hazy picture was processed to improve contrast. In normal picture, the crane cables, background forest , exact shape of two square buildings and dark lines on the left foreground brown building cannot be recognized. In the Registrator output they can. See also Example on Wikimedia Commons

Quadruples the megapixels of the camera

Resolution quadrupling is not perfect because frequencies approaching the maximum one (pixel values 1,0,1,0,1,0,1,...) are lost in the camera. It is also 4 times slower than registration - works on upsampled images.

Removes JPEG artifacts

Removes noise from dark details

Digital effects don't degrade anymore

Misty Zurich was histogram equalized, gamma adjusted and filtered with Retinex. The normal image shows noisy meadows and red artifact over church dial. Twibright Registrator produces perfectly smooth meadows.

Hides people

Removes stripes on CRT's

Removes rain

Long exposure in daylight

Couple minutes of simulated exposure.

Download and install

Tested on Linux, should work on Mac, BSD. For Windows you need Cygwin.

Make sure djpeg from libjpeg is installed by running djpeg --help. If not, install libjpeg.
Make sure libpng is installed in a way that programs can compile against it. Usual names on Linux distributions with packages: libpng-dev, libpng-devel, png-dev, png-devel
Download securely via HTTPS (provided by Internet Archive): Registrator Tar Gzip archive version 20230122
Unpack: tar xvf registrator_20230122.tgz
Enter the source: cd registrator
make
su and type in the root password
make install

How to use Registrator

Shoot images

Set ISO for noise in the image. Stronger than the JPEG artifacts, but not comparable with the image itself.
If not using tripod and delayed/remote trigger, the exposure time should be fast, but not faster that the camera limit. Minimize motion blur from camera vibration. Otherwise, the photo gets defocused.
The more shots the smoother result. Try to not shift the camera between images too much. Otherwise, if the lens warps, blur might occur. The signal to noise ratio grows with square root of image count.

Register images for lower noise

Copy JPEGs from the camera or convert RAW format into PPMs. Run registrator with filenames as parameters. Example:

registrator p100????.jpg p101????.ppm

Registrator will print a lot of matching progress info. When it ends, you have file registrator_output.png. It is a 16-bit depth PNG containing the registered output. It also makes partial results - for example - registrator_intermediate_7.png - which contains the first 7 images registered together. This is useful if the registration derails.

Quadruple resolution

This makes sense only if the camera lens is sharper than the resolution of the sensor. It depends on iris setting and tripod. It takes 4 times longer than registering.

Convert camera images to twice the size (scaling 200%) with nearest neighbor - without interpolation. example:
```
convert in.jpg -scale 200% -interpolate nearest-neighbor out.ppm
```
Run registrator on the ppm's like when registering. This is going to take 4 times longer than when registering!

Convert registrator_output.png into ppm:

convert registrator_output.png registrator_output.ppm

Deconvolve:
```
deconvolve 3 < registrator_output.ppm
```
Examine the result:
```
gimp registrator_deconvolved.png
```
If the result is too blurred, decrease the deconvolve parameter towards 2. If it's too fringy, increase the number. Repeat the deconvolution until you are satisfied with the image.

Dehaze

convert registrator_output.png registrator_output.ppm

If you don't have ImageMagick, you can use GIMP.

Load registrator_output.png into image editor and crop part of the image you want to dehaze. Save as part.ppm.
Let black and white levels be automatically determined:
```
deconvolve 2 < part.ppm
```
This prints 6 numbers example: 18 2 3 223 227 245

deconvolve 18 2 3 223 227 245 < registrator_output.ppm

Floating point numbers are accepted. No hexadecimal.

Examine the result:
```
gimp registrator_deconvolved.png
```

Register images for 3D or video stabilization

Copy JPEGs from the camera or convert RAW format into PPMs. Run registrator with OVERWRITE=1 and filenames as parameters. Example:

OVERWRITE=1 registrator p100????.jpg p101????.ppm

Registrator will print a lot of matching progress info. Let's assume you have 100 files on input. Then Registrator produces registrator_intermediate_1.png to registrator_intermediate_99.png and registrator_output.png which is the last frame.

Troubleshooting

Error during make, undefined reference to png_create_write_struct

Make sure libpng is installed on the system including the header files (png.h). Maybe you need png-dev, libpng-dev, png-devel or libpng-devel package. Various Linux distributions install libraries into various places. There is no standard for that. Maybe some library is in an unexpected location. Please tell me the error so I can try to fix it.

The result looks sharpened, noisy, or blurred

This happens when the count of input images is low. Use more photographs. Registrator uses deconvolution to cancel blurring caused by many photographings and resamplings of the same picture with fractional pixel shifts. However, with small amount of pictures, this blurring depends on exact random relative positioning of the pictures and varies within the image field. It would be very difficult to calculate and deblur this. Therefore an average blurring is calculated when removing the blurring. This can lead to slight sharpening or blurring depending on luck.

Multiple copies of images in the output or part of the image black

Happens when there are more ways how to match or the pictures differ too much. Example: roof or pavement with repeating tiles, people or cars obscuring large portions of the picture, too much noise in the picture. In case of repeating structures use wider view to include parts which are not ambiguous to match and then crop the undesired away in image editor. In case of people or cars discard pictures which have large portion obscured. In case of noise select longer exposure. You may need a tripod for that

Look into registrator_intermediate_*.png to see the progess of the registration and possibly throw away problematic files. The number in the filename says how many files registered that file contains.

JPEG artifacts still recognizable in the output

Happens when the noise in the input images is too low. Some level of noise is necessary to break through the JPEG roundings. Use higher ISO to get more noise.

Vertical or horizontal smears across the whole picture

Happens when heavy effects are used on the output. This is the CCD smear due to imperfect readout. The smears are in tows and columns with areas. Use an effect that isn't so aggressive. Try a better quality camera.

Grainy or different structured edges

Happens when the pictures shift between exposures. The edges are then averaged from only those pictures that contain them. Crop the output or shoot again with tripod.

I want to use raw input, not JPEGs!

Look inside the registrator script. The registrator-bin executable itself takes concatenated PPM (bit depth 8bits) on the input. Convert your raw pictures to PPM and feed registrator-bin manually.

Fringes in the resolution quadrupled image

Happens when the strength of the deconvolution is closer to 2. Use higher number in the deconvolution.

Small letters illegible in resolution quadrupled image

The finest details are already lost in the camera when the resolution quadrupled image was photograped. They cannot be recovered anymore. Letters with 1-pixel wide lines spaced 1 pixel contain this finest detail and are weakened or obliterated.

Happens when the strength of the deconvolution is closer to 2. Use higher number in the deconvolution.

How does it work?

How registration works

Twibright Registrator contains a high precision high performance bilinear interpolation engine working with 32-bit float pixel depth. Using this engine, it tries to find shift, rotation, zoom and exposure change so that the next picture matches the previous one with as little average pixel error as possible. It adds these pictures together and applies an adaptive deconvolution to cancel the blurring caused by stacking many pictures photographed with random fractional pixel offsets. If the count of photos were large, this deconvolution exactly cancels the blur. The work is done in photometrically linear space to ensure physical equivalency - the procedure is equivalent to a shakeless camera which exposes for very long time, gathering the photons, and then counting them.

How resulution quadrupling works

Let'sa imagine the real picture consists of pixels 2 times finer than the camera has. In 1D situation, The camera always takes 2 pixels together (if it's precisely aligned) or 1 pixel and parts of 2 other pixels. Now we assume the pixel offsets are uniformly distributed. Let's take a sample image 0,0,0,1,0,0,0 and calculate resulting images when many such images (upsampled by nearest neighbor after being photograped) are registered. In case the examined pixel falls within the big pixel, it produces the core 1,2,1. In the other case, it produces core 2,5,6,5,2. If the 1st core is scaled to the same sum 7,14,7 and the cores added, we get 2,12,20,12,2, or 1,6,10,6,1. This is a convolution of two cores: 1,2,1 and 1,4,1. The 1,4,1 is actually caused by the registration and the 1,2,1 by smearing always two pixels together. 1,4,1 is already removed by the registrator. So we have to remove 1,2,1. In comparison to 1,4,1 it's impossible to remove completely since some frequencies (the highest) is zeroed out so we would have to divide by zero. So we take a near core like 1,2.5,1 or 1,3,1 and remove that one. The particular middle number of the core has to be tried out manually to get the best tradeoff between fringes and blurriness.

License?

GPL 3 or later. The example photos are under GFDL or CC-BY-SA.

Changelog

April 16, 2014: Doesn't compile on Ubuntu 10.04. Fixed.
June 29, 2009: Deconvolution strength is now adjusted according to the number of processed photos.
June 30, 2009: Writes registrator_output.png directly using libpng instead of using ImageMagick
July 1, 2009: Registrator now accepts any PPM bit depth.
July 4, 2009: Produces intermediate pngs for debugging.
February 1, 2010: Speeded up to 193% of original speed times by optimizing the C code manually. Added presettable pixel precision.

Author?

Registrator was written in June 2009 by Karel Kulhavy.