{"id":21427059,"url":"https://github.com/japhir/corepics","last_synced_at":"2025-07-05T06:37:21.692Z","repository":{"id":56406855,"uuid":"311330778","full_name":"japhir/corepics","owner":"japhir","description":"Plot IODP core photographs in R with ggplot2 and ggtextures","archived":false,"fork":false,"pushed_at":"2020-11-10T16:30:55.000Z","size":647,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-23T07:44:57.302Z","etag":null,"topics":["core","data-science","drill","geology","ggplot2","iodp","magick","palaeoclimate","palaeontology","paleoclimate","paleontology","photo","plotting","r"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/japhir.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-09T12:25:55.000Z","updated_at":"2023-03-01T07:14:53.000Z","dependencies_parsed_at":"2022-08-15T18:10:21.909Z","dependency_job_id":null,"html_url":"https://github.com/japhir/corepics","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/japhir%2Fcorepics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/japhir%2Fcorepics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/japhir%2Fcorepics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/japhir%2Fcorepics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/japhir","download_url":"https://codeload.github.com/japhir/corepics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243933425,"owners_count":20370989,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["core","data-science","drill","geology","ggplot2","iodp","magick","palaeoclimate","palaeontology","paleoclimate","paleontology","photo","plotting","r"],"created_at":"2024-11-22T21:43:46.937Z","updated_at":"2025-03-16T21:20:31.268Z","avatar_url":"https://github.com/japhir.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"[[file:1411_corepics.png]]\n\n#+TITLE: CorePics\n#+property: header-args:R  :session *R:corepics* :exports both :results output :eval no-export\n\nThis is how I use R, some command line utilities, and imagemagick to get IODP core photographs in a reduced size into my scientific figures!\n\n* what you need\n- [[R][R]]\n- [[https://www.gnu.org/software/bash/][bash]] or another command line shell\n- [[https://imagemagick.org/][imagemagick]]\n- the develpment R package [[https://github.com/clauswilke/ggtextures][ggtextures]]\n\n** R libraries\n#+begin_src R\n  library(stringr)\n  library(dplyr)\n  library(glue)\n  library(readxl)\n  library(readr)\n  ## library(ggtextures)\n  devtools::load_all(\"~/Downloads/ggtextures\")\n#+end_src\n\n#+RESULTS:\n\n: Loading ggtextures\n\n* download the images and section depth\nIODP info is available on [[http://iodp.tamu.edu/][the IODP website]]\n\n- Core data from all ODP, DSDP and IODP expedition 301--312 data are available on the [[http://iodp.tamu.edu/janusweb/imaging/photo.shtml][janus database]]\n  - For older core images (before leg 198) we need the url: http://www-odp.tamu.edu/publications/###_IR/VOLUME/CORES/IMAGES to get PDFs of the whole core, sections laid out next to each other.\n  - There is also a newer file format called MrDIT, but this was too hard to get to work.\n- Core data for all newer IODP expeditions from 317--present are available in the [[http://web.iodp.tamu.edu/LORE/][LORE database]].\n\n** get image files/links from the janus database\nIn this example setup we're looking at Expedition 208, Site 1264. After some digging around on the janus database, we find that the JPG images are hosted with a regular url:\nhttp://www-odp.tamu.edu/publications/208_IR/VOLUME/CORES/JPEG/1264B/1264b_026h/1264b_026h_06.jpg\n\nThese are the ones I'll be using.\n\nThis function converts leg, site, hole, etc. data to create the URL to the image.\n#+begin_src R\n  create_image_url \u003c- function(leg, site, hole, core, type = \"H\", section, image_type = \"JPEG\", extension = \".jpg\") {\n    base_url \u003c- \"http://www-odp.tamu.edu/publications/\"\n    core_padded \u003c- str_pad(core, 3, pad = 0)\n    type_l \u003c- str_to_lower(type)\n    sec \u003c- section %\u003e% str_to_lower() %\u003e% str_pad(2, pad=0)\n    glue(\"{base_url}{leg}_IR/VOLUME/CORES/{image_type}/{site}{hole}/{site}{str_to_lower(hole)}_{core_padded}{type_l}/{site}{str_to_lower(hole)}_{core_padded}{type_l}_{sec}{extension}\")\n  }\n\n  # some tests\n  create_image_url(leg = 208, site = 1264, hole = \"B\", core = 30, type = \"H\", section = \"CC\")\n  create_image_url(leg = 208, site = 1264, hole = \"C\", core = 1, type = \"H\", section = 2)\n  create_image_url(leg = 208, site = 1264, hole = \"B\", core = 26, type = \"H\", section = 6)\n#+end_src\n\n#+RESULTS:\n\n: http://www-odp.tamu.edu/publications/208_IR/VOLUME/CORES/JPEG/1264B/1264b_030h/1264b_030h_cc.jpg\n: http://www-odp.tamu.edu/publications/208_IR/VOLUME/CORES/JPEG/1264C/1264c_001h/1264c_001h_02.jpg\n: http://www-odp.tamu.edu/publications/208_IR/VOLUME/CORES/JPEG/1264B/1264b_026h/1264b_026h_06.jpg\n\n*** get core/section/depth info\nThis section is probably something that you need to tweak some more for your use-case.\n\nIn the end, you need the IODP core info up to the section level. Each sections hould have a ~top_depth~ (in mbsf, mcd, or armcd) and ~bot_depth~.\n\n#+begin_src R\n  # this is the affine table, with the recovery and adjusted depths of the cores\n  md \u003c- read_excel(\"~/SurfDrive/PhD/sites/site_1264_walvisridge/208 1264 Composite.xlsx\",\n                   sheet = \"Affine Table\", range = \"A1:S31\") %\u003e% # hole A\n    # I combine the info for the three holes\n    bind_rows(read_excel(\"~/SurfDrive/PhD/sites/site_1264_walvisridge/208 1264 Composite.xlsx\",\n                         sheet = \"Affine Table\", range = \"A33:S63\")) %\u003e% # hole B\n    bind_rows(read_excel(\"~/SurfDrive/PhD/sites/site_1264_walvisridge/208 1264 Composite.xlsx\",\n                         sheet = \"Affine Table\", range = \"A65:S66\")) # hole C\n\n  # now I need the recovered depth per section from somewhere\n  sd \u003c- read_tsv(\"~/SurfDrive/PhD/programming/iodp_read_info/data/208-1264/sections/208_1264_sections.txt\",\n                 skip = 2) %\u003e%\n    rename_all(str_to_lower)\n\n  # combine the metadata and clean up\n  cm \u003c- left_join(sd, md, by = c(\"expedition\" = \"Leg\",\n                                 \"site\" = \"Site\",\n                                 \"hole\" = \"H\",\n                                 \"core\" = \"Cor\",\n                                 \"core_type\" = \"T\")) %\u003e%\n    mutate(armcd_top = `New Cum Offset` + top_depth,\n           armcd_bot = `New Cum Offset` + bottom_depth) %\u003e%\n    select(expedition:core_type, section_label, top_depth, rev_length, armcd_top, armcd_bot) %\u003e%\n    # here we create the image url based on the metadata\n    mutate(url = create_image_url(leg=expedition,\n                                  site=site,\n                                  hole=hole,\n                                  core=core,\n                                  type=core_type,\n                                  section=section_label,\n                                  image_type=\"JPEG\"))\n#+end_src\n\n*** download the images so I can crop them\nWe create a directory where we save all the pretty large jpg images. For walvis ridge, this takes quite a while: 4 minutes and 30 seconds to download 455 files.\n\nThis is the part where you have to have bash or another shell with the cat, awk, grep, and wget utilities installed.\n#+begin_src R\n  dir \u003c- \"1264_corepics\"\n  dir.create(dir)\n\n  # save the clean metadata to a tsv so we can use awk to work with it\n  write_tsv(cm, \"cm.txt\")\n\n  # download the corepics\n  system(\"cat cm.txt | awk '{ print $11 }' | grep '^http' | wget -P 1264_corepics -i-\")\n#+end_src\n\n#+RESULTS:\n\nFINISHED --2020-11-09 12:02:14--\nTotal wall clock time: 5m 48s\nDownloaded: 455 files, 1.2G in 4m 30s (4.57 MB/s)\n\n** get core/section/depth info from the LORE database\nIf you are working with a newer core, good on you! It's much easier to get those data in shape!\n\n1. Go to http://web.iodp.tamu.edu/LORE/ in a modern browser\n2. In the left menu, click on \"images\"\n3. click \"Core Sections (LSIMG)\"\n4. click \"standard\"\n5. in the filtering view, filter by your site/core/section as desired\n6. optional: click an alternate depth scale\n7. click \"View data\"\n8. click \"Download tabular data\" to save the csv with the metadata\n9. click \"Batch download linked files\" to save the jpg images\n10. Choose \"cropped images\"\n\nWe can merge the file paths with the metadata by their image link:\n\n#+begin_src R :eval never\n  cp \u003c- read_csv(\"dat/core_site_pictures.csv\")\n\n  # list all the downloaded jpg files\n  fl \u003c- tibble(file = list.files(\"dat/corepics\", pattern=\".jpg\")) %\u003e%\n    # I'm assuming that these ID's are unique here\n    mutate(id = str_extract(file, \"[0-9]+.jpg$\") %\u003e% str_replace(\".jpg\", \"\") %\u003e% parse_double())\n\n  cp \u003c- cp %\u003e%\n    # merge the file info (section depth etc.) with the image name\n    left_join(fl, by = c(\"Cropped image (JPG) link\" = \"id\")) %\u003e%\n    mutate(file = str_replace(file, \".jpg\", \".png\"))\n#+end_src\n\nThe image at the top is the result of reading in and combining data for IODP site U1411 using this method! Don't forget the next cropping step though, or your computer might crash.\n\n** resize and crop the images\nThe images are much too large to load into memory all at once, so we downsize and crop them all.\n\nThe target width at 300 dpi if we want to plot it at half a cm:\n#+begin_src R\n 300 / 2.54 * .5 # 300 dpi in cm, for half a cm\n#+end_src\n\n#+RESULTS:\n\n: [1] 59.05512\n\nUse magick's mogrify to batch resize and crop the images. Play around with the magick commandline options to get the cropping correct.\n#+begin_src R\n  system(\"mogrify -resize 60 -crop -17-54 -format png 1264_corepics/*.jpg\")\n#+end_src\n\n** add file paths to small images\n#+begin_src R\n  cm \u003c- cm %\u003e%\n    mutate(file = paste0(\"1264_corepics/\", basename(url) %\u003e% str_replace(\".jpg\", \".png\"))) %\u003e%\n    # some of the sections don't have images, or our download failed perhaps?\n    mutate(file_exists = !is.na(file.info(file)$size),\n           file = ifelse(file_exists, file, NA_character_))\n#+end_src\n\n* plot the smaller images on the correct locations\n#+begin_src R :results output graphics file :file 1264_corepics.png :width 200 :height 1600\n  cm %\u003e%\n    ggplot(aes(y = armcd_bot, xmin = 0L, xmax = 1L, ymin = armcd_bot, ymax = armcd_top, image = file)) +\n    geom_rect(alpha = .2) +\n    ## coord_cartesian(ylim = c(17, 3)) +\n    geom_textured_rect(colour = NA, nrow = 1, ncol = 1, img_width = unit(1, \"null\"), img_height = unit(1, \"null\"), interpolate = FALSE) +\n    facet_grid(cols = vars(hole)) +\n    scale_y_reverse() +\n    labs(title=\"IODP Leg 208 Site 1264\", subtitle = \"core photographs\", caption = \"created by Ilja Kocken\") +\n    theme(axis.title.x = element_blank(), axis.text.x = element_blank(), axis.ticks.x = element_blank(),\n          panel.grid = element_blank()) -\u003e pl\n  pl\n#+end_src\n\n#+RESULTS:\n\n[[file:1264_corepics.png]]\n\n* combine the core photographs with your figures\nTo do this, I recommend the patchwork package. Make sure to set the exact same y-axis for both plots.\n\n#+begin_src R\n  cm %\u003e%\n    ggplot(aes(y = armcd_bot, xmin = 0L, xmax = 1L, ymin = armcd_bot, ymax = armcd_top, image = file)) +\n    geom_rect(alpha = .2) +\n    coord_cartesian(ylim = c(17, 3)) +\n    geom_textured_rect(colour = NA, nrow = 1, ncol = 1, img_width = unit(1, \"null\"), img_height = unit(1, \"null\"), interpolate = FALSE) +\n    facet_grid(cols = vars(hole)) +\n    scale_y_reverse() +\n    theme(axis.title.x = element_blank(), axis.text.x = element_blank(), axis.ticks.x = element_blank(),\n          panel.grid = element_blank()) -\u003e pl\n\n  cm %\u003e%\n    mutate(D47 = rnorm(n(), mean = 0.76, sd = 0.2)) %\u003e%\n    ggplot(aes(x = D47, y = armcd_bot)) +\n    scale_y_reverse() +\n    coord_cartesian(ylim = c(17, 3)) +\n    theme(axis.title.y = element_blank(), axis.text.y = element_blank()) +\n    geom_point() -\u003e pl2\n#+end_src\n\n#+begin_src R :results output graphics file :file 1264_corepics_with_data.png :width 500 :height 600\n  library(patchwork)\n\n  pc \u003c- (pl + pl2) + plot_layout(widths = c(.1, .9))\n  pc\n#+end_src\n\n#+RESULTS:\n\n[[file:1264_corepics_with_data.png]]\n\n** putting the depth on the x-axis\nIf you need to put the depth on the x-axis, the images need to be rotated. Make sure you rotate them in the correct way! If you want to put greater depths to the right, rotate the images 90° anti-clockwise, like so:\n\n#+begin_src R :eval never\n  system(\"mogrify -rotate -90 1264_corepics/*.png\")\n#+end_src\n\n#+RESULTS:\n\nIf you want to put the deeper sediments to the left, so that time progresses from left to right, rotate them by +90°.\n\n#+begin_src R\n  system(\"mogrify -rotate 90 1264_corepics/*.png\")\n#+end_src\n\n#+RESULTS:\n\n#+begin_src R\n  cm %\u003e%\n    ggplot(aes(x = armcd_bot, ymin = 0L, ymax = 1L, xmin = armcd_bot, xmax = armcd_top, image = file)) +\n    geom_rect(alpha = .2) +\n    coord_cartesian(xlim = c(17, 3)) +\n    geom_textured_rect(colour = NA, nrow = 1, ncol = 1, img_width = unit(1, \"null\"), img_height = unit(1, \"null\"), interpolate = FALSE) +\n    facet_grid(rows = vars(hole)) +\n    ## scale_x_reverse() +\n    theme(axis.title.y = element_blank(), axis.text.y = element_blank(), axis.ticks.y = element_blank(),\n          panel.grid = element_blank()) -\u003e pl\n\n  cm %\u003e%\n    mutate(D47 = rnorm(n(), mean = 0.76, sd = 0.2)) %\u003e%\n    ggplot(aes(y = D47, x = armcd_bot)) +\n    scale_y_reverse() +\n    coord_cartesian(xlim = c(17, 3)) +\n    theme(axis.title.x = element_blank(), axis.text.x = element_blank()) +\n    geom_point() -\u003e pl2\n#+end_src\n\n#+RESULTS:\n\n#+begin_src R :results output graphics file :file 1264_corepics_with_data_on_x-axis.png :width 600 :height 500\n  pc \u003c- (pl2/pl) + plot_layout(heights = c(.9, .1))\n  pc\n#+end_src\n\n#+RESULTS:\n\n[[file:1264_corepics_with_data_on_x-axis.png]]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaphir%2Fcorepics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaphir%2Fcorepics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaphir%2Fcorepics/lists"}