ivis dimensionality reduction

ivis is a machine learning library for reducing dimensionality of very large datasets using Siamese Neural Networks. ivis preserves global data structures in a low-dimensional space, adds new data points to existing embeddings using a parametric mapping function, and scales linearly to millions of observations. The algorithm is described in detail in Structure-preserving visualisation of high dimensional single-cell datasets.

Features

• Unsupervised, semi-supervised, and fully supervised dimensionality reduction

• Support for arbitrary datasets

  • N-dimensional arrays
  • Image files on disk
  • Custom data connectors

• In- and out-of-memory data processing

• Resumable training

• Arbitrary neural network backbones

• Customizable neighbour retrieval

• Callbacks and Tensorboard integration

The latest development version is on github.

Get Started

• Python Package
  • Installation
  • Dependencies
  • Quickstart
  • Bugs
• R Package
  • Installation
  • Quickstart
  • Vignette

Using ivis

• Unsupervised Dimensionality Reduction
  • The iris dataset
  • Reducing dimensionality of n-dimensional arrays
  • Using custom KNN retreaval
• Supervised Dimensionality Reduction
  • Classification
  • Regression
  • Supervision Weight
• Semi-supervised Dimensionality Reduction
  • Semi-supervised Classification
  • Supervision Weight
• Hyperparameter Selection
  • k
  • n_epochs_without_progress
  • model

Examples

• Ivis Notebooks

Applications

• Visualising Single Cell Experiments
  • Reducing Dimensionality Using ivis
• Dimensionality Reduction
  • Visual Assessment
  • Quantitative Evaluation
• Metric Learning
  • Introduction
  • Results
  • Conclusions
• Out-of-memory Datasets
  • Introduction
  • Example
  • Conclusions
  • Advanced: Custom dataset loaders

Benchmarks

• Speed of Execution
  • Effects of Data Size on Performance
• Distance Preservation
  • Datasets Selection
  • Accuracy of Low-Dimensional Embeddings

API Reference

• Ivis
• Neighbour Retrieval
• Indexable Datasets
• Losses
• Callbacks