Data can be linked from a fileserver to OMERO. This page explains how to do it and how to set it up for your institute fileserver.

Importing images from the CAi daten fileserver

Organizing your data in the dedicated folder

On the filesever \\cai.ad.hhu.de\daten, you must place your data in a dedicated folder that can be accessed by OMERO:

Z:\OMERO_in-place_import\AG_Xyz\Firstname_Lastname_HHUusername

Your group folders are created ahead and managed by the CAi. If you are not able to see them, let your admin know by email (tom.boissonnet@hhu.de) or via https://rocketchat.hhu.de 

You can arrange your folders then in whichever way you like. Note however that once imported, YOU SHOULD NOT RENAME any file or parent folder (this would break the link to OMERO).

We thus highly recommend you adopt a good organization from the start. Get in touch with us if you wish to discuss it ahead (ksenia.krooss@hhu.de , tom.boissonnet@hhu.de).

Two recommendation:

• Avoid nesting your folders too much. Data is organized in OMERO, and OMERO can tell you where the images are. Too many subfolders will just make it too hard to navigate
• Consistently name your folders. Start with the date YYYY-MM-DD, no space, not too long name, information from most generic to most specific:

2024-07-31_Arabidopsis_Mutation-XYZ_sample-1

Getting authorized to import data

The import is only made possible for those having access to the folders containing data. In your group folder in "OMERO_in-place_import\AG_Xyz", create the file "allowed_users.txt" (or edit it if it already exists) and add your username to the list (one HHU username per line).

Anyone in that list will be able to import the data located in subfolders.

Triggering the import from OMERO.web

Connect to https://omero-cai.hhu.de and open the script "import_script/Fileserver Import..."

"Data Type" and "IDs" are filled automatically from the selected container before starting the script.

• To generate Datasets inside a Project according to folder names, select the project in OMERO
• To attach all imported images to a single Dataset, select that dataset in OMERO

• For High-Content Screening data, select the screen to which you want to attach the generated plate

"Main folder path" is the root of what you will import. Subsequent subfolders may be used to name your datasets. From Windows File Explorer, select the full path by clicking right to the last folder name in the address bar:

File path will correspond to the relative path from the "main folder path" to the images you wish to import. For our above example, selecting the highlighted path as "main folder path":

• *.czi    will import all images with czi extension in the "main folder path"
• */image_3.czi      will import the image named "image_3.czi" in all subfolder from the "main folder path"
• *.czi, details/*.czi      will import all czi images from "main folder path" and all czi images in details folder. (multiple path are provided separated by a comma ","). This can be used to import multiple folders from your user folder.

In the third scenario above, two Datasets will be created (if a Project was selected as Data Type). One named "2024-07-31_testfolder" with image_1.czi and image_2.czi, and a second named "2024-07-31_testfolder__details" with images from the details folder.

Three more options:

• Skip Min/Max: this will instruct the import not to search for min and max pixel values in your images. Will speed up import, but you may need to adjust the pixel range yourself depending on the image file format.
• Skip Thumbnail: Useful for High-Content-Screening (in case of multiple fields per well). Will instruct the import not to generate thumbnails for your images. Will speed up the import, but thumbnail will be generated the first time you request it in OMERO.web. 
• Dry run: This will print the script output as usual (which files are found, what datasets will be generated, ...) but without importing the images; useful if you are unsure of how it works;

Importing images from your institute fileserver

In this case, your institute fileserver needs to be linked to OMERO. A dedicated user account for OMERO should be granted Read-Only access by the ZIM to the fileserver or a dedicated folder on the fileserver. 

Once OMERO can access data on the fileserver, this will be configured by a CAi admin to have your fileserver listed in the script (Filserver name). 

Most rules and explanations given above will also apply to your fileserver. Just select your fileserver when importing data instead of "cai_daten"

FAQ

• I have images on OMERO, how do I know if they are linked to the fileserver?

Look for the Fileset information of the image. If it is linked to the fileserver, you will see the information "--transfer=ln_s", as well as the path to the image on the fileserver (from an OMERO's standpoint, your own path will be slightly different)

• Can I make existing images in OMERO linked to the fileserver?

This is not possible. Images should be reimported with the script, and old images deleted from OMERO. NOTE: figures you made or annotations like tags or key-value pairs will not automatically be changed to the new images. We thus advise only to manage new images this way.

• Is it different for OMERO to have images imported this way?

There are no differences in working with images in OMERO from "normal import" VS "fileserver import" (OMERO follows a link to the files on the fileserver instead of having the images itself). However, importing from the fileserver grants you the possibility to access the raw files.

• If I delete my image in OMERO, does it delete the file on the fileserver too?

No. OMERO will only delete the link and not the original file (it's a symbolic link).

• Is my data safe on the fileserver?

Fileserver and OMERO storage have the same backup policy from the ZIM. Duplicating data from one to the other does not grant additional safety to your data, but only occupies more space on our storage (billing incoming, 300€ per TB)

• I renamed a file or folder and now OMERO cannot view the image.

OMERO will still list your images in their datasets, with their thumbnails, annotations, etc. However, for anything that involves reading the raw pixels (iviewer, OMERO.figure), OMERO will not be able to read your data anymore. Look at the Fileset information to find the original name and location of the file to revert your changes back.

• Is it faster to import large images this way

What is faster is that the images are not uploaded to the server. However, if your images are large in X and Y (>3000), OMERO still needs to process subresolutions for each plane (that can increases rapidly when considering multiple channels, timepoints are Z-slices). For faster import, you have to generate the pyramidal resolutions and import the resulting images (remember to discard the previous image to not occupy twice the storage space). This can most easily be done for free with https://www.glencoesoftware.com/products/ngff-converter/   See Convert to ome.tiff wiki page (Conversion of image to ome.tiff before import in OMERO)