tvercaut

Soft (partial-volume) label assignment for nanodrr

Prototype accompanying eigenvivek/nanodrr#92, now also compared with the maintainer's soft-labels branch (render(..., soft_labels=True)), which the notebook installs.

In nanodrr 0.1.6, render_torch routes each ray sample to a per-structure channel with a nearest-neighbour label lookup. This gist prototypes soft labels: the one-hot labelmap is interpolated trilinearly, giving, per class, the intensity integral and a projected soft label (the path length in mm that each ray spends in the class). Nothing in the library is modified.

• soft_vs_nearest_labels.ipynb (executed; open in Colab): the renderers, figures, and a benchmark that runs on every available torch device.
• Z_build_nb.py:

tvercaut

grid_sample on MPS: what is missing, and plain-PyTorch workarounds

TL;DR

With torch >= 2.13 every F.grid_sample kernel nanodrr uses has a native MPS implementation (forward and backward, 2D and 3D, all modes), and nothing here is needed. The native backward is ~10x faster than the best plain-PyTorch backward below. The workarounds only matter for torch <= 2.12 (nanodrr currently pins 2.11).

tvercaut

tvercaut

Basic attempt at injecting and reading back user data unregistered (UDU) supplemental enhancement information (SEI) with ffmpeg / libav.

Some references:

• https://ffmpeg.org/doxygen/trunk/group__lavu__frame.html#ggae01fa7e427274293aacdf2adc17076bca4f2dcaee18e5ffed8ff4ab1cc3b326aa
• https://gist.github.com/figgis/d3cff69be815491eff1ac9338aa78add
• https://stackoverflow.com/q/74431792

tvercaut

#!/usr/bin/env python3

import numpy as np
import matplotlib.pyplot as plt

def wavelength_to_rgb(wavelengths, gamma=0.8):
    '''This converts a given wavelength of light to an
    approximate RGB color value. The wavelength must be given
    in nanometers in the range from 380 nm through 750 nm
    (789 THz through 400 THz).

tvercaut

A simple PyTorch implementation of plane fitting with RANSAC. Some ideas are taken from: https://github.com/leomariga/pyRANSAC-3D.

In each (batched-)RANSAC iteration, a batch of triplets is sampled and assessed in parallel. A cross-product is used to get plane equations from triplets. The refinement, post identification of inliers, is done with an eigen decomposition of P.T @ P where P is the matrix of points in homogeneous coordinates. Classicially, an SVD decomposition of P is used but this can be slow and memory intensive with large number of points. We thus opted for the eigen decomposition approach despite it being less numerically stable.

tvercaut

Hack file to get a nicer Gist name

tvercaut

import numpy as np
import matplotlib.pyplot as plt
import imageio
url = 'https://archive.org/download/10thframekixxc64800dpi48bit/10th%20Frame%20%28Kixx%29_C64_Cassette_800dpi_48bit.tif'
im0 = imageio.imread(url)
print('im0:',np.min(im0),np.max(im0),im0.shape,im0.dtype)

im0gray = 0.299*np.float32(im0[:,:,0]) + 0.587*np.float32(im0[:,:,1]) + 0.114*np.float32(im0[:,:,2])
im0gray = im0gray / np.max(im0gray)
im1 = np.uint16(np.round( im0gray*((1<<10)-1) ))

tvercaut

Spectral information

https://omlc.org/spectra/hemoglobin/summary.html https://omlc.org/spectra/icg/

tvercaut

#EXTM3U
#EXTINF:0,Radio France - FIP (AAC+Metadata)
http://radio-metadata.fr:8000/fip.m3u
#EXTINF:0,Radio France - FIP Reggae (AAC+Metadata)
http://radio-metadata.fr:8000/fipreggae.m3u
#EXTINF:0,Radio France - FIP Cultes (AAC+Metadata)
http://radio-metadata.fr:8000/fipcultes.m3u
#EXTINF:0,Radio France - FIP World (AAC+Metadata)
http://radio-metadata.fr:8000/fipmonde.m3u
#EXTINF:0,Radio France - FIP Nouveautés (AAC+Metadata)