{"id":15442810,"url":"https://github.com/miguelcarcamov/csromer","last_synced_at":"2025-10-04T10:54:01.413Z","repository":{"id":39855729,"uuid":"220523278","full_name":"miguelcarcamov/csromer","owner":"miguelcarcamov","description":"Compressive Sensing and Optimization Framework to reconstruct Faraday Depth signals","archived":false,"fork":false,"pushed_at":"2023-07-10T15:55:03.000Z","size":155930,"stargazers_count":6,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-25T13:01:39.609Z","etag":null,"topics":["astronomy-astrophysics","astrophysics","compressed-sensing","faraday-depth","faraday-rotation","faraday-tomography","framework","linear-polarization","magnetic-fields","object-oriented","object-oriented-programming","python","signal-reconstruction"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miguelcarcamov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-08T18:14:06.000Z","updated_at":"2025-03-17T13:08:21.000Z","dependencies_parsed_at":"2025-03-03T00:42:23.454Z","dependency_job_id":null,"html_url":"https://github.com/miguelcarcamov/csromer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/miguelcarcamov/csromer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miguelcarcamov%2Fcsromer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miguelcarcamov%2Fcsromer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miguelcarcamov%2Fcsromer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miguelcarcamov%2Fcsromer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miguelcarcamov","download_url":"https://codeload.github.com/miguelcarcamov/csromer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miguelcarcamov%2Fcsromer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277240004,"owners_count":25785111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-27T02:00:08.978Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["astronomy-astrophysics","astrophysics","compressed-sensing","faraday-depth","faraday-rotation","faraday-tomography","framework","linear-polarization","magnetic-fields","object-oriented","object-oriented-programming","python","signal-reconstruction"],"created_at":"2024-10-01T19:30:23.974Z","updated_at":"2025-10-04T10:54:01.397Z","avatar_url":"https://github.com/miguelcarcamov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CS-ROMER\n\n*Compressed Sensing ROtation MEasure Reconstruction*\n\nCompressed sensing reconstruction framework for Faraday depth spectra.\nPlease feel free to open an issue if you spot a bug. This is an open source project, and therefore you can fork, make changes and submit a [pull request](https://github.com/miguelcarcamov/csromer/pulls) of any of your additions and modifications.\n\n- This paper explains what is [Faraday rotation measure synthesis](https://www.aanda.org/articles/aa/abs/2005/39/aa2990-05/aa2990-05.html)\n- Wikipedia information about [Faraday effect](https://en.wikipedia.org/wiki/Faraday_effect)\n\n## Features\n\n- Simulation of Faraday depth sources\n- Subtraction of Galactic RM\n- Reconstruction of Faraday depth sources from linearly polarized data\n- Reconstruction of Faraday depth sources using Compressed Sensing\n- More than 100 wavelet filters provided by `Pywavelets`\n\nThis code will run in a Python \u003e= 3.9.7 environment with all the packages installed (see `requirements.txt` file).\n\n## Examples\n\nExamples and use of cases can be found [here](https://github.com/miguelcarcamov/cs-romer-notebooks)\n\n## Citing\n\nThe paper of this software is under submission but if you use it you can cite it as:\n\n```tex\n@article{10.1093/mnras/stac3031,\n    author = {Cárcamo, Miguel and Scaife, Anna M M and Alexander, Emma L and Leahy, J Patrick},\n    title = \"{CS-ROMER: A novel compressed sensing framework for Faraday depth reconstruction}\",\n    journal = {Monthly Notices of the Royal Astronomical Society},\n    year = {2022},\n    month = {10},\n    abstract = \"{The reconstruction of Faraday depth structure from incomplete spectral polarization radio measurements using the RM Synthesis technique is an under-constrained problem requiring additional regularisation. In this paper we present cs-romer: a novel object-oriented compressed sensing framework to reconstruct Faraday depth signals from spectro-polarization radio data. Unlike previous compressed sensing applications, this framework is designed to work directly with data that are irregularly sampled in wavelength-squared space and to incorporate multiple forms of compressed sensing regularisation. We demonstrate the framework using simulated data for the VLA telescope under a variety of observing conditions, and we introduce a methodology for identifying the optimal basis function for reconstruction of these data, using an approach that can also be applied to datasets from other telescopes and over different frequency ranges. In this work we show that the delta basis function provides optimal reconstruction for VLA L-band data and we use this basis with observations of the low-mass galaxy cluster Abell 1314 in order to reconstruct the Faraday depth of its constituent cluster galaxies. We use the cs-romer framework to de-rotate the Galactic Faraday depth contribution directly from the wavelength-squared data and to handle the spectral behaviour of different radio sources in a direction-dependent manner. The results of this analysis show that individual galaxies within Abell 1314 deviate from the behaviour expected for a Faraday-thin screen such as the intra-cluster medium and instead suggest that the Faraday rotation exhibited by these galaxies is dominated by their local environments.}\",\n    issn = {0035-8711},\n    doi = {10.1093/mnras/stac3031},\n    url = {https://doi.org/10.1093/mnras/stac3031},\n    note = {stac3031},\n    eprint = {https://academic.oup.com/mnras/advance-article-pdf/doi/10.1093/mnras/stac3031/46643343/stac3031.pdf},\n}\n```\n\n## Installation\n\nThe software can be installed as a python package locally or using Pypi\n\n### Locally after cloning the project\n\n```shell\ngit clone https://github.com/miguelcarcamov/csromer.git\ncd csromer\npip install .\n```\n\n### Locally as developer\n\n```shell\ngit clone git@github.com:miguelcarcamov/csromer.git\ncd csromer\npip install -e .\n```\n\nWe highly recommend installing [pre-commit](https://pre-commit.com) to develop over this code.\nThis will allow you to run hooks that reformat the project files according to our style.\n\n### From PyPI\n\n`pip install csromer`\n\n### From Github\n\n`pip install -U git+https://github.com/miguelcarcamov/csromer.git`\n\n### From latest docker container\n\n`docker pull ghcr.io/miguelcarcamov/csromer:latest`\n\n## Simulate Faraday sources directly in frequency space\n\nCS-ROMER is able to simulate Faraday depth spectra directly in wavelength-squared space. The classes `FaradayThinSource` and `FaradayThickSource` inherit directly from `Dataset`, and therefore you can directly use them as an input to your reconstruction.\n\n### Thin sources\n\n```python\nimport numpy as np\nfrom csromer.simulation import FaradayThinSource\n# Let's create an evenly spaced frequency vector from 1.008 to 2.031 GHz (JVLA setup)\nnu = np.linspace(start=1.008e9, stop=2.031e9, num=1000)\n# Let's say that the peak polarized intensity will be 0.0035 mJy/beam with a spectral index = 1.0\npeak_thinsource = 0.0035\n# The Faraday source will be positioned at phi_0 = -200 rad/m^2\nthinsource = FaradayThinSource(nu=nu, s_nu=peak_thinsource, phi_gal=-200, spectral_idx=1.0)\n```\n\n### Thick sources\n\n```python\nimport numpy as np\nfrom csromer.simulation import FaradayThickSource\n# Let's create an evenly spaced frequency vector from 1.008 to 2.031 GHz (JVLA setup)\nnu = np.linspace(start=1.008e9, stop=2.031e9, num=1000)\n# Let's say that the peak polarized intensity will be 0.0035 mJy/beam with a spectral index = 1.0\npeak_thicksource = 0.0035\n# The Faraday source will be positioned at phi_0 = 200 rad/m^2 and will have a width of 140 rad/m^2\nthicksource = FaradayThickSource(nu=nu, s_nu=peak_thicksource, phi_fg=140, phi_center=200, spectral_idx=1.0)\n```\n\n### Simulate\n\nOnce you have set your source parameters, you can call the `simulate()` function as\n\n```python\nthinsource.simulate()\nthicksource.simulate()\n```\n\nThis call will simulate the linealy polarized emission and it will assign the data to the `data` attribute.\n\n### Mixed sources\n\nA thin+thick or mixed source is simply a superposition/sum of a thin source and thick source. Therefore we have overriden the `+` operator in order to sum these two objects.\n\n```python\nmixedsource = thinsource + thicksource\n```\n\nThe result will be a `FaradaySource` object.\n\n### Remove frequency channels randomly as if you were doing RFI flagging\n\nThe framework also allows you to randomly remove data with the function `remove_channels` to simulate RFI flagging\n\n```python\n# Let's say that we want to randomly remove 20% of the data\nmixedsource.remove_channels(0.2)\n```\n\n### Adding noise to your simulations\n\nIf we want to add random Gaussian noise to our simulation we can simply call the function `apply_noise`\n\n```python\n# Let's add Gaussian random noise with mean 0 and standard deviation equal\n# to 20% the peak of the signal.\nsigma = 0.2*mixedsource.s_nu\nmixedsource.apply_noise(sigma)\n```\n\n## Reconstruct 1D Faraday sources\n\nTo illustrate how to reconstruct Faraday depth signals with CS-ROMER first we will reconstruct the mixed source that we have just constructed\n\n### Dirty Faraday depth spectra\n\n```python\nfrom csromer.reconstruction import Parameter\nfrom csromer.transformers import DFT1D\n# We first need to initialize the parameter object that will contain our Faraday depth\n# data either in Faraday-depth space or in wavelet space\nparameter = Parameter()\n# We calculate the cellsize in Faraday depth space using an oversampling factor of 8\n# Here parameter.data is set as a complex array of zeros\nparameter.calculate_cellsize(dataset=mixedsource, oversampling=8)\n# We instantiate our discrete Fourier transform\ndft = DFT1D(dataset=mixedsource, parameter=parameter)\n# We calculate the dirty Faraday depth spectra\nF_dirty = dft.backward(mixedsource.data)\n```\n\n### Reconstruct simulated data\n\n```python\nfrom csromer.transformers import NUFFT1D\n# We instantiate our non-uniform FFT\nnufft = NUFFT1D(dataset=mixedsource, parameter=parameter, solve=True)\n# At this point we can use either the parameter data set with zeros or we can\n# use the dirty Faraday depth spectra\nparameter.data = F_dirty\nparameter.complex_data_to_real() # We convert the complex data to real\n# You can set the L1 lambda regularization manually or estimate it as\nlambda_l1 = np.sqrt(mixedsource.m + 2*np.sqrt(mixedsource.m)) * np.sqrt(2) * np.mean(mixedsource.sigma)\n```\n\n### Objective function\n\n```python\nfrom csromer.objectivefunction import L1, Chi2\nfrom csromer.objectivefunction import OFunction\n# We instantiate each part of our objective function\nchi2 = Chi2(dft_obj=nufft, wavelet=None) # chi-squared\nl1 = L1(reg=lambda_l1) # L1-norm regularization\n\nF_obj = OFunction([chi2, l1]) # Whole objective function\nf_obj = OFunction([chi2]) # Only chi-squared\ng_obj = OFunction([l1]) # Just regularizations\n```\n\n### Optimization algorithm\n\nOne of the ways to optimize the objective function is to use the FISTA algorithm.\n\n```python\nfrom csromer.optimization import FISTA\n# We instantiate our FISTA object as\nopt = FISTA(guess_param=parameter, F_obj=F_obj, fx=chi2, gx=g_obj, noise=mixedsource.theo_noise, verbose=False)\n# We run the optimization algorithm\nobj, X = opt.run()\nX.real_data_to_complex() # We convert the data back to complex when the optimization finishes\n```\n\nThis returns the objective function value `obj` and `X`a `Parameter` instance object. Therefore in this case `X.data` will hold the reconstructed Faraday depth spectra.\nAt this point you can also access to the model and residual data in wavelength-squared as `mixedsource.model_data` and `mixedsource.residual`, respectively. You can calculate the residuals in Faraday depth space by using the DFT object as\n\n```python\nF_residual = dft.backward(mixedsource.residual)\n```\n\n### Using discrete or undecimated wavelets\n\nCS-ROMER has about 100 filters to user with discrete wavelet transforms or undecimated wavelet transforms. We use the `Pywavelets` package, for more information please refer to [PyWavelets](https://pywavelets.readthedocs.io/en/latest/index.html). To use the wavelets in cs-romer you can do:\n\n```python\nfrom csromer.dictionaries import DiscreteWavelet, UndecimatedWavelet\n# This line instantiates a discrete wavelet\nwav = DiscreteWavelet(wavelet_name=\"coif3\", mode=\"periodization\", append_signal=False)\n# This line instantiates an undecimated wavelet\nwav = UndecimatedWavelet(wavelet_name=\"sym2\", mode=\"periodization\", append_signal=True)\n```\n\nThe `append_signal` parameter plugs the Faraday depth spectrum to your coefficients resulting in redundancy in your coefficients. If you just want the wavelet coefficients then set `append_signal=False`.\nAt this point our parameter object data needs to be our coefficients and not our Faraday depth spectra, therefore, we do\n\n```python\nparameter.data = F_dirty # Suppose that you set your parameter data with your dirty Faraday depth spectrum\nparameter.complex_data_to_real() # We convert the data to real\n# Here we do a wavelet decomposition of our Faraday depth space\n# We set the coefficients of the decomposition as our parameter data\nparameter.data = wav.decompose(parameter.data)\n# Don't forget to change your chi-squared\nchi2 = Chi2(dft_obj=nufft, wavelet=wav)\n```\n\nYou might have noticed that at the end of the optimization we will end up with fitted coefficients instead of a Faraday depth spectrum.\nTherefore, we need to reconstruct the Faraday depth spectrum from our coefficients doing\n\n```python\nX.data = wav.reconstruct(X.data) # We reconstruct the Faraday depth spectrum from coefficients\nX.real_data_to_complex() # We convert the real Faraday depth spectrum into complex\n```\n\n### Reconstruct a real line of sight data\n\nTo reconstruct real data your main script should follow the same workflow. The only difference is that you need to instantiate a `Dataset` object.\n\n```python\nfrom csromer.base import Dataset\n# nu is the irregular spaced frequency\n# data is the polarized emission\n# sigma is the error per channel (this can be an array of ones or rms calculation per image channel)\n# alpha is the spectral index at this line of sight\ndataset = Dataset(nu=nu, data=data, sigma=sigma, spectral_idx=alpha)\n```\n\n### Subtracting the Milky Way RM contribution\n\nWe use [S. Hutschenreuter et al.](https://www.aanda.org/articles/aa/full_html/2022/01/aa40486-21/aa40486-21.html) Faraday sky HealPIX image to subtract the galactic RM contribution at a certain position of the sky using the object `FaradaySky`.\nNote that you can omit this step, and subtract any RM value that you might find appropiate.\n\n```python\nfrom csromer.faraday_sky import FaradaySky\nfrom astropy.coordinates import SkyCoord\nimport astropy.units as un\n\nf_sky = FaradaySky()\ncoord = SkyCoord(ra=173.694*un.deg, dec=48.957*un.deg, frame=\"fk5\")\ngal_mean, gal_std = f_sky.galactic_rm(coord.ra, coord.dec, frame=\"fk5\")\ndataset.subtract_galacticrm(gal_mean.value)\n```\n\n## Reconstruct a cube\n\nWe warn the users that not all framework functions are yet implemented to work with data cubes. Therefore, we need to use `numpy` broadcasting and the package `joblib`. Let's say that you have read your polarized cube and frequency array using `np.load`. For this example we will assume that you will reconstruct with uniform weights.\n\n```python\nimport numpy as np\nfrom csromer.reconstruction import Parameter\nfrom csromer.base import Dataset\nfrom joblib import Parallel, delayed\n\nQU_cubes = np.load('qu_cubes.npy') # Shape (freqs, m, n)\nnu = np.load('nu.npy') # Shape (freqs,)\nm = QU_cubes.shape[1]\nn = QU_cubes.shape[2]\n\nQ = QU_cubes[0]\nU = QU_cubes[1]\ndata = Q + 1j * U\nsigma = np.ones_like(nu) # Uniform weights\n# We will construct a dataset only to obtain Faraday-space array shapes\nfoo_dataset = Dataset(nu=nu, sigma=sigma, spectral_idx=0.0)\nfoo_parameter = Parameter()\nparameter.calculate_cellsize(dataset=foo_dataset, oversampling=8)\n# Faraday dispersion function cube\n# Note that ee add another dimension to store dirty, model, residual and restored signals\nF = np.zeros(4, foo_parameter.n, m, n, dtype=np.complex64)\n\n# Parallelize your for loop using joblib\ntotal_pixels = m*n\nnthreads = 8\nworkers_1d_idxs = np.arange(total_pixels)\nworkers_idxs = np.unravel_index(workers_1d_idxs, (M,N))\nParallel(n_jobs=nthreads, backend=\"multiprocessing\", verbose=10)(delayed(reconstruct_cube)(\n        F, data, sigma, nu, 0.0, workers_idxs, i, eta, False) for i in range(0, total_pixels))\n```\n\n```python\ndef reconstruct_cube(F=None, data=None, sigma=None, nu=None, spectral_idx=None, noise=None,\n                     workers_idxs=None, idx=None, eta=1.0, use_wavelet=True):\n    i = workers_idxs[0][idx]\n    j = workers_idxs[1][idx]\n\n    if spectral_idx is None:\n        spectral_idx = 0.0\n\n    dataset = Dataset(nu=nu, sigma=sigma, data=data[:, i, j], spectral_idx=spectral_idx)\n    parameter = Parameter()\n    parameter.calculate_cellsize(dataset=dataset, oversampling=8, verbose=False)\n\n    dft = DFT1D(dataset=dataset, parameter=parameter)\n    nufft = NUFFT1D(dataset=dataset, parameter=parameter, solve=True)\n\n    F_dirty = dft.backward(dataset.data)\n\n    # We can estimate the noise from the edges of the FDF\n    edges_idx = np.where(np.abs(parameter.phi) \u003e parameter.max_faraday_depth / 1.5)\n    noise = eta * 0.5 * (np.std(F_dirty[edges_idx].real) + np.std(F_dirty[edges_idx].imag))\n\n    # We store the FDF\n    F[0, :, i, j] = F_dirty\n\n    # Let's say that if use_wavelet is True then we use the coif2 wavelet\n    if use_wavelet:\n        wav = UndecimatedWavelet(wavelet_name=\"coif2\")\n    else:\n        wav = None\n\n    # We estimate lambda for L1 norm\n    lambda_l1 = np.sqrt(2 * len(dataset.data) + np.sqrt(4 * len(dataset.data))) * noise\n    chi2 = Chi2(dft_obj=nufft, wavelet=wav)\n    l1 = L1(reg=lambda_l1)\n    F_func = [chi2, l1]\n    f_func = [chi2]\n    g_func = [l1]\n\n    F_obj = OFunction(F_func)\n    g_obj = OFunction(g_func)\n\n    parameter.data = F_dirty\n    parameter.complex_data_to_real()\n\n    if use_wavelet:\n        parameter.data = wav.decompose(parameter.data)\n\n    opt = FISTA(guess_param=parameter, F_obj=F_obj, fx=chi2, gx=g_obj, noise=noise, verbose=False)\n    obj, X = opt.run()\n\n    if use_wavelet:\n        X.data = wav.reconstruct(X.data)\n\n    X.real_data_to_complex()\n    F_residual = dft.backward(dataset.residual)\n    F[1, :, i, j] = X.data\n    F[2, :, i, j] = X.convolve(normalized=True) + F_residual\n    F[3, :, i, j] = F_residual\n```\n\nNote that if your Faraday depth cube is large, then probably it won't fit in your memory. Therefore, we can use `memory map`. In that case you would need to define your Faraday depth cube as:\n\n```python\noutput_file_mmap = os.path.join(folder, 'output_mmap')\nF = np.memmap(output_file_mmap, dtype=np.complex64, shape=(4, foo_parameter.n, M, N), mode='w+')\n```\n\n## Contact\n\nPlease if you have any problem, issue or you catch a bug using this software please use the [issues tab](https://github.com/miguelcarcamov/csromer/issues) if you have a common question or you look for any help please use the [discussions tab](https://github.com/miguelcarcamov/csromer/discussions).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiguelcarcamov%2Fcsromer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiguelcarcamov%2Fcsromer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiguelcarcamov%2Fcsromer/lists"}