1. Configuring the Giotto Environment

Giotto contains several functions that contain wrappers to Python code and thus requires an environment containing Python. Utilizing the functionality of the reticulate package, Giotto contains a function which sets up a miniconda environment and installs the required Python packages within that environment. Once this function, installGiottoEnvironment, has been run, Giotto will automatically default to utilizing this environment.

2. Start Giotto

# Ensure Giotto Suite is installed
if(!"Giotto" %in% installed.packages()) {
  pak::pkg_install("drieslab/Giotto")
}

library(Giotto)

# Ensure Giotto Data is installed
if(!"GiottoData" %in% installed.packages()) {
  pak::pkg_install("drieslab/GiottoData")
}

library(GiottoData)
# Ensure the Python environment for Giotto has been installed
genv_exists = checkGiottoEnvironment()
if(!genv_exists){
  # The following command need only be run once to install the Giotto environment
  installGiottoEnvironment()
}

The function installGiottoEnvironment two particular arguments that are most useful for reinstallation, if necessary:

  • force_miniconda: force reinstallation of miniconda, default is FALSE
  • force_environment: force reinstallation of the Giotto environment, default is FALSE

Note that, by default, installGiottoEnvironment installs a specific version of Python and each required package. At the time of this tutorial’s creation, the following versions are utilized:

3. Customizing the Giotto Installation

If different versions of Python or packages are necessary for a workflow, Giotto may be installed accordingly. Ensure that all required packages, which have been listed above, are accounted for when installing. Simply specify the desired version numbers for each package within a vector, and provide that vector to the packages_to_install argument within installGiottoEnvironment.

Note that machine type is not relevant when providing packages_to_install to installGiottoEnvironment; this function will identify the OS in use and install/not install packages (i.e. python.app) accordingly.

### Note that the following code has been provided to indicate how to install
### Giotto with customized Python and Python package versions. It has been 
### intentionally commented out so that it will not run and overwrite the 
### default versions unless deliberately edited.

### new_pkg_versions <- c('pandas==1.4.4',
###                       'networkx==2.6.3',
###                       'python-igraph==0.9.6',
###                       'leidenalg==0.8.7',
###                       'python-louvain==0.15',
###                       'scikit-learn==0.24.2',
###                       'python.app==2')
### 
### ############################
### # If altering the original Giotto Installation is not desired, DO NOT
### # run the following command as written.
### ############################
### installGiottoEnvironment(packages_to_install = new_pkg_versions,
###                          python_version = '3.8') # Default is 3.10.2

4. Advanced: Using a non-default Conda Environment with Giotto

If using reticulate's default miniconda path to create an environment is undesirable, the Giotto environment may be created within an existing anaconda/miniconda environment by specifying the mini_install_path argument:

installGiottoEnvironment(mini_install_path = "C:/my/conda/lives/here")

If not provided, it is chosen by reticulate::install_miniconda(). Please note the required input format: - Correct format: mini_install_path = “C:/my/conda/lives/here” OR “C:\my\conda\lives\here” - INCORRECT formats: mini_install_path = “C:/my/conda/lives/here/” AND “C:\my\conda\lives\here\”

Unexpected behavior could arise if force_miniconda is set to TRUE when mini_install_path is specified and encompasses a non-reticulate environment, as this prompts a reticulate miniconda installation.

Note that the installation of all aforementioned packages is necessary for the full functionality of Giotto. A .yml file is provided in the repository for convenience of alternative installation methods. If the desired environment is not named “giotto_env”, Giotto will be unable to automatically detect the conda environment, so it must be specified within a workflow. To use a specific, non-default named Conda environment, the path to a system-specific python executable within that environment must be provided to createGiottoInstructions. This will direct reticulate to activate and utilize that environment within that R session. See How to Create a Giotto Object for more details.

5. Package Accessibility

Giotto makes use of the following Python packages (and their respective dependencies) for full functionality:

Here is a brief troubleshooting workflow to investigate if reticulate can access them.

Note that “community” and “sklearn” are aliases of “python-louvain” and “scikit-learn”, respectively.

# Creating Giotto Instructions without specifying a Python path will make 
# reticulate activate the default Giotto environment. 
default_instrs <- createGiottoInstructions()

# Extract python path information
default_python_path <- default_instrs$python_path

# Make reticulate iteratively check for the packages
pkg_check <- function(){
  py_pkgs = c('pandas','networkx', 'igraph', 'leidenalg','community','sklearn','python.app')
  py_pkg_error = character()
  test_availability = TRUE
  
  for (i in py_pkgs){
    if(i == 'python.app' & Sys.info()[['sysname']] != "Darwin"){
      # If the machine OS is not OSX (Mac), break out of the loop
      # Otherwise, also check for python.app
      break
    }
    test_availability <- reticulate::py_module_available(i)
    if(!test_availability) {py_pkg_error <- c(py_pkg_error,i)}
  }
  
  if(test_availability){
    cat('All Python packages for Giotto are accessible at environment:\n', default_python_path)
  }else{
    for (x in py_pkg_error) cat(x,'was not found within environment:\n',default_python_path,'\n\n')
  }
  
  return(py_pkg_error)
}

pkg_check()

6. Troubleshooting Packages not Found

# Restart the R session, while maintaining workspace variables.
# If using RStudio, the following command will do exactly that:
.rs.restartR()

# Direct reticulate to use Python within the Giotto Environment
reticulate::use_python(default_python_path)

# Check if packages exist again. Ensure function from above code block is defined.
missing_packages <- pkg_check()

retry_install <- length(missing_packages) > 0

if(retry_install){
  
  # Attempt to reinstall all packages.
  pkgs_w_versions <- c('pandas==1.5.1',
                       'networkx==2.8.8',
                       'python-igraph==0.10.2',
                       'leidenalg==0.9.0',
                       'python-louvain==0.16',
                       'python.app==1.4',
                       'scikit-learn==1.1.3')
  
  python_version = "3.10.2"
  
  py_pkgs = c('pandas','networkx',
              'igraph', 'leidenalg',
              'python-louvain','scikit-learn','python.app')
  
  if(Sys.info()[['sysname']] != "Darwin"){
    pkgs_w_versions = pkgs_w_versions[!grepl(pattern = 'python.app', x = pkgs_w_versions)]
    py_pkgs = py_pkgs[!grepl(pattern = 'python.app', x = py_pkgs)]
  }
  
  env_location <- reticulate::py_discover_config()$pythonhome
  partial_path_to_conda <- paste0(reticulate::miniconda_path(),'/envs/giotto_env')
  
  py_lou = pkgs_w_versions[grepl(pattern = 'python-louvain',x = pkgs_w_versions)]
  pip_packages = c("smfishhmrf", py_lou)
  pkgs_w_versions = pkgs_w_versions[!grepl(pattern = 'python-louvain',x = pkgs_w_versions)]
  
  if(.Platform[['OS.type']] == 'unix') {
    
    conda_full_path = paste0(partial_path_to_conda,'/','bin/conda')
    
    # Remove all previous installations
    reticulate::conda_remove(envname = env_location,
                             packages = py_pkgs,
                             conda = conda_full_path)
    
    # Reinstall
    reticulate::conda_install(packages = pkgs_w_versions,
                              envname = env_location,
                              method = 'conda',
                              conda = conda_full_path,
                              python_version = python_version)
    
    # Reinstall with pip
    reticulate::conda_install(packages = pip_packages,
                              envname = env_location,
                              method = 'conda',
                              conda = conda_full_path,
                              pip = TRUE,
                              python_version = python_version)
  }
  else if(.Platform[['OS.type']] == 'windows'){
    conda_full_path = paste0(partial_path_to_conda,'/','condabin/conda.bat')
    
    # Remove all previous installations
    reticulate::conda_remove(envname = env_location,
                             packages = py_pkgs,
                             conda = conda_full_path)
    
    # Reinstall
    reticulate::conda_install(packages = pkgs_w_versions,
                              envname = env_location,
                              method = 'conda',
                              conda = conda_full_path,
                              python_version = python_version,
                              channel = c('conda-forge', 'vtraag'))
    
    # Reinstall with pip
    reticulate::conda_install(packages = pip_packages,
                              envname = env_location,
                              method = 'conda',
                              conda = conda_full_path,
                              pip = TRUE,
                              python_version = python_version)
  }
}

If this does not fix the issue at hand, here are some potential action items:

  • Remove and attempt to reinstall the Giotto environment.
  • Run removeGiottoEnvironment, then terminate R.
  • Open a completely new R session, and run installGiottoEnvironment
  • Post to an issue to the Giotto GitHub page here.
  • Please include the version numbers of R, Giotto, and the OS in use at the time of the issue.

7. Session Info

R version 4.3.2 (2023-10-31)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Sonoma 14.3.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Giotto_4.0.2      GiottoClass_0.1.3

loaded via a namespace (and not attached):
 [1] SummarizedExperiment_1.32.0 gtable_0.3.4                rjson_0.2.21               
 [4] xfun_0.42                   ggplot2_3.4.4               Biobase_2.62.0             
 [7] lattice_0.22-5              vctrs_0.6.5                 tools_4.3.2                
[10] bitops_1.0-7                generics_0.1.3              parallel_4.3.2             
[13] stats4_4.3.2                tibble_3.2.1                fansi_1.0.6                
[16] colorRamp2_0.1.0            pkgconfig_2.0.3             Matrix_1.6-5               
[19] checkmate_2.3.1             data.table_1.15.0           S4Vectors_0.40.2           
[22] lifecycle_1.0.4             GenomeInfoDbData_1.2.11     compiler_4.3.2             
[25] GiottoUtils_0.1.5           munsell_0.5.0               terra_1.7-71               
[28] codetools_0.2-19            GenomeInfoDb_1.38.6         htmltools_0.5.7            
[31] RCurl_1.98-1.14             yaml_2.3.8                  pillar_1.9.0               
[34] crayon_1.5.2                SingleCellExperiment_1.24.0 DelayedArray_0.28.0        
[37] magick_2.8.2                abind_1.4-5                 gtools_3.9.5               
[40] tidyselect_1.2.0            digest_0.6.34               dplyr_1.1.4                
[43] rprojroot_2.0.4             fastmap_1.1.1               grid_4.3.2                 
[46] here_1.0.1                  colorspace_2.1-0            cli_3.6.2                  
[49] SparseArray_1.2.3           magrittr_2.0.3              S4Arrays_1.2.0             
[52] utf8_1.2.4                  withr_3.0.0                 backports_1.4.1            
[55] scales_1.3.0                rmarkdown_2.25              XVector_0.42.0             
[58] matrixStats_1.2.0           reticulate_1.35.0           GiottoVisuals_0.1.4        
[61] png_0.1-8                   SpatialExperiment_1.12.0    evaluate_0.23              
[64] knitr_1.45                  GenomicRanges_1.54.1        IRanges_2.36.0             
[67] rlang_1.1.3                 Rcpp_1.0.12                 glue_1.7.0                 
[70] BiocGenerics_0.48.1         rstudioapi_0.15.0           jsonlite_1.8.8             
[73] R6_2.5.1                    MatrixGenerics_1.14.0       zlibbioc_1.48.0