de.mpg.biochem.mars.molecule

Interface MoleculeArchive<M extends Molecule,I extends MarsMetadata,P extends MoleculeArchiveProperties<M,I>,X extends MoleculeArchiveIndex<M,I>>

Type Parameters:

M - Molecule type.

I - MarsMetadata type.

P - MoleculeArchiveProperties type.

All Superinterfaces:

JsonConvertibleRecord

All Known Implementing Classes:

AbstractMoleculeArchive, DefaultMoleculeArchive, DnaMoleculeArchive, ObjectArchive, SingleMoleculeArchive

public interface MoleculeArchive<M extends Molecule,I extends MarsMetadata,P extends MoleculeArchiveProperties<M,I>,X extends MoleculeArchiveIndex<M,I>>
extends JsonConvertibleRecord

MoleculeArchives are the primary storage structure of Mars datasets. MoleculeArchives provides an optimal structure for storing single-molecule time-series data. Time-series data for each molecule in a dataset are stored in the form of Molecule records, which may also contain calculated parameters, tags, notes, and kinetic change point segments. These records are assigned a UID string at the time of creation. This string provides univeral molecule uniqueness throughout all datasets. MoleculeArchives contain a collection of molecule records associated with a given experimental condition or analysis pipeline.

MarsMetadata records containing data collection information are also stored in MoleculeArchives. They are identified using metaUID strings. Molecule records associated with a given data collection have a metaUID string linking them to the correct MarsMetadata record within the same MoleculeArchive. Global properties of the MoleculeArchive, including indexing, comments, etc.., are stored in a MoleculeArchiveProperties record also contained within the MoleculeArchive.

See AbstractMoleculeArchive for further information.

Author:

Karl Duderstadt

Method Summary

Modifier and Type	Method and Description
void	addMoleculeTags(java.util.Map<java.lang.String,java.lang.String> tagMap) Add tags to molecules using UID to tag map.
boolean	contains(java.lang.String UID) Used to check if there is a molecule record with the UID given.
boolean	containsMetadata(java.lang.String UID) Used to check if there is a metadata record with the UID given.
X	createIndex() Create empty MoleculeArchiveIndex.
X	createIndex(com.fasterxml.jackson.core.JsonParser jParser) Create empty MoleculeArchiveIndex using the JsonParser stream given.
I	createMetadata(com.fasterxml.jackson.core.JsonParser jParser) Create MarsMetadata record using JsonParser stream.
I	createMetadata(java.lang.String metaUID) Create empty MarsMetadata record with the metaUID specified.
M	createMolecule() Create empty Molecule record.
M	createMolecule(com.fasterxml.jackson.core.JsonParser jParser) Create Molecule record using the JsonParser stream given.
M	createMolecule(java.lang.String UID) Create empty Molecule record with the UID specified.
M	createMolecule(java.lang.String UID, MarsTable table) Create Molecule record using the UID and MarsTable specified.
P	createProperties() Create empty MoleculeArchiveProperties record.
P	createProperties(com.fasterxml.jackson.core.JsonParser jParser) Create MoleculeArchiveProperties record using JsonParser stream.
void	deleteMetadatasWithTag(java.lang.String tag) Removes all metadata records with the tag provided.
void	deleteMoleculesWithTag(java.lang.String tag) Removes all molecule records with the tag provided.
M	get(int index) Retrieves the molecule record at the provided index.
M	get(java.lang.String UID) Get the molecule record with the given UID.
int	getChannel(java.lang.String UID) Channel for the molecule with the given UID.
java.lang.String	getComments() Global comments.
java.io.File	getFile() Returns the File from which the archive was opened.
int	getImage(java.lang.String UID) Image index for the molecule with the given UID.
I	getMetadata(int index) Retrieves an MarsMetadata record.
I	getMetadata(java.lang.String metaUID) Retrieves a metadata record.
java.lang.String	getMetadataTagList(java.lang.String UID) Comma separated list of tags for the metadata record with the given UID.
java.util.Set<java.lang.String>	getMetadataTagSet(java.lang.String UID) Tags for the metadata record with the given UID.
java.lang.String	getMetadataUIDforMolecule(java.lang.String UID) Get the UID of the metadata for a molecule record.
java.util.List<java.lang.String>	getMetadataUIDs() Retrieves the list of UIDs of all metadata records.
java.util.List<double[][]>	getMoleculeBatch(java.util.List<java.lang.String> UIDs, java.lang.String tColumn, java.lang.String signalColumn, int rangeStart, int rangeEnd, java.util.List<java.lang.String> tagsToLearn, int threads) Utility function to generate batches of molecules data in an optimal format for machine learning using keras.
java.util.List<java.lang.String>	getMoleculeUIDs() Retrieves the list of UIDs for all Molecule records.
java.lang.String	getName() Get the name of the archive.
int	getNumberOfMetadatas() Number of metadata records in the MoleculeArchive.
int	getNumberOfMolecules() Number of molecule records in the MoleculeArchive.
java.lang.String	getTagList(java.lang.String UID) Comma separated list of tags for the molecule with the given UID.
java.util.Set<java.lang.String>	getTagSet(java.lang.String UID) Tags for the molecule with the given UID.
MoleculeArchiveWindow	getWindow() Returns the MoleculeArchiveWindow holding this archive, if one exists.
boolean	isVirtual() True if the archive is virtual, false if not.
void	log(java.lang.String str) Add a log message to all MarsImageMetadata records.
void	logln(java.lang.String str) Add a log message to all MarsImageMetadata records.
java.util.stream.Stream<I>	metadata() Convenience method to retrieve a metadata stream.
boolean	metadataHasTag(java.lang.String UID, java.lang.String tag) Check if a MARSImageMetadata record has a tag.
boolean	moleculeHasNoTags(java.lang.String UID) Check if a molecule record has no tags.
boolean	moleculeHasTag(java.lang.String UID, java.lang.String tag) Check if a molecule record has a tag.
boolean	moleculeHasTags(java.lang.String UID) Check if a molecule record has tags.
java.util.stream.Stream<M>	molecules() Convenience method to retrieve a Molecule stream.
java.util.Set<java.lang.String>	moleculeTags(java.lang.String UID) Retrieve the list of tags for a molecule.
java.util.stream.Stream<I>	parallelMetadata() Convenience method to retrieve a multithreated Molecule stream.
java.util.stream.Stream<M>	parallelMolecules() Convenience method to retrieve a multithreaded Molecule stream.
MoleculeArchiveProperties<M,I>	properties() Get the MoleculeArchiveProperties which contain general information about the archive.
void	put(M molecule) Adds a molecule to the archive.
void	putMetadata(I metadata) Adds an MarsMetadata record to the archive.
void	rebuildIndexes() Rebuild all indexes by inspecting the contents of store directories.
void	rebuildIndexes(int nThreads) Rebuild all indexes by inspecting the contents of store directories.
void	remove(M molecule) Removes the molecule record provided.
void	remove(java.lang.String UID) Removes the molecule record with the given UID.
void	removeMetadata(I meta) The metadata record given is removed from the archive.
void	removeMetadata(java.lang.String metaUID) The metadata record with the UID given is removed from the archive.
void	save() Saves the MoleculeAchive to the file from which it was opened.
java.io.File	saveAs(java.io.File file) Saves MoleculeAchive to the given file destination in smile format.
java.io.File	saveAsJson(java.io.File file) Saves MoleculeAchive to the given file destination in json format.
void	saveAsJsonVirtualStore(java.io.File virtualDirectory) Creates the directory given and a virtual store inside with all files in json format and with .json file extensions.
void	saveAsJsonVirtualStore(java.io.File virtualDirectory, int nThreads) Creates the directory given and a virtual store inside with all files in json format with .json file extension.
void	saveAsVirtualStore(java.io.File virtualDirectory) Creates the directory given and a virtual store inside.
void	saveAsVirtualStore(java.io.File virtualDirectory, int nThreads) Creates the directory given and a virtual store inside with all files in smile format with .sml file extension.
void	setComments(java.lang.String comments) Sets the global comments.
void	setFile(java.io.File file) Set the file the archive should save to.
void	setName(java.lang.String name) Set the name of the archive.
void	setWindow(MoleculeArchiveWindow win) Set the window containing this archive.

Methods inherited from interface de.mpg.biochem.mars.molecule.JsonConvertibleRecord

dumpJSON, fromJSON, getJsonGenerator, getJsonParser, setJsonField, setShowWarnings, toJSON

Method Detail

rebuildIndexes

void rebuildIndexes()
             throws java.io.IOException

Rebuild all indexes by inspecting the contents of store directories. Then save the new indexes to the indexes.json file in the store.

Throws:

java.io.IOException - if something goes wrong saving the indexes.

rebuildIndexes

void rebuildIndexes(int nThreads)
             throws java.io.IOException

Rebuild all indexes by inspecting the contents of store directories. Then save the new indexes to the indexes.json file in the store. Use the number of threads specified.

Parameters:

nThreads - The thread count.

Throws:

java.io.IOException - if something goes wrong saving the indexes.

save

void save()
   throws java.io.IOException

Saves the MoleculeAchive to the file from which it was opened.

Throws:

java.io.IOException - if something goes wrong saving the data.

saveAs

java.io.File saveAs(java.io.File file)
             throws java.io.IOException

Saves MoleculeAchive to the given file destination in smile format.

Parameters:

file - a yama file destination. If the .yama is not present it will be added.

Returns:

File with correct extensions where archive was saved.

Throws:

java.io.IOException - if something goes wrong saving the data.

saveAsJson

java.io.File saveAsJson(java.io.File file)
                 throws java.io.IOException

Saves MoleculeAchive to the given file destination in json format.

Parameters:

file - a yama.json file destination. If the .yama.json is not present it will be added.

Returns:

File with correct extensions where archive was saved.

Throws:

java.io.IOException - if something goes wrong saving the data.

saveAsVirtualStore

void saveAsVirtualStore(java.io.File virtualDirectory)
                 throws java.io.IOException

Creates the directory given and a virtual store inside. Rebuilds indexes in the process if the archive was loaded from a virtual store.

Parameters:

virtualDirectory - a directory destination for the virtual store.

Throws:

java.io.IOException - if something goes wrong creating the virtual store.

saveAsVirtualStore

void saveAsVirtualStore(java.io.File virtualDirectory,
                        int nThreads)
                 throws java.io.IOException

Creates the directory given and a virtual store inside with all files in smile format with .sml file extension. This is the default format. Rebuilds indexes in the process if the archive was loaded from a virtual store.

Parameters:

virtualDirectory - a directory destination for the virtual store.

nThreads - The thread count.

Throws:

java.io.IOException - if something goes wrong creating the virtual store.

saveAsJsonVirtualStore

void saveAsJsonVirtualStore(java.io.File virtualDirectory)
                     throws java.io.IOException

Creates the directory given and a virtual store inside with all files in json format and with .json file extensions. Rebuilds indexes in the process if the archive was loaded from a virtual store.

Parameters:

virtualDirectory - a directory destination for the virtual store.

Throws:

java.io.IOException - if something goes wrong creating the virtual store.

saveAsJsonVirtualStore

void saveAsJsonVirtualStore(java.io.File virtualDirectory,
                            int nThreads)
                     throws java.io.IOException

Creates the directory given and a virtual store inside with all files in json format with .json file extension. Rebuilds indexes in the process if the archive was loaded from a virtual store.

Parameters:

virtualDirectory - a directory destination for the virtual store.

nThreads - The thread count.

Throws:

java.io.IOException - if something goes wrong creating the virtual store.

put

void put(M molecule)

Adds a molecule to the archive. If a molecule with the same UID is already in the archive, the record is updated. All indexes are updated with the properties of the molecule added.

Parameters:

molecule - a record to add or update.

putMetadata

void putMetadata(I metadata)

Adds an MarsMetadata record to the archive. If an MarsMetadata record with the same UID is already in the archive, the record is updated. All indexes are updated with the properties of the MarsMetadata record added.

Parameters:

metadata - an metadata record to add or update.

removeMetadata

void removeMetadata(java.lang.String metaUID)

The metadata record with the UID given is removed from the archive. All indexes are updated to reflect the change.

Parameters:

metaUID - the UID of the metadata record to remove.

removeMetadata

void removeMetadata(I meta)

The metadata record given is removed from the archive. All indexes are updated to reflect the change.

Parameters:

meta - metadata record to remove.

getMetadata

I getMetadata(int index)

Retrieves an MarsMetadata record.

Parameters:

index - The index of the MarsMetadata record to retrieve.

Returns:

A MarsMetadata record.

getMetadata

I getMetadata(java.lang.String metaUID)

Retrieves a metadata record.

Parameters:

metaUID - The UID of the metadata record to retrieve.

Returns:

A metadata record.

getMetadataUIDs

java.util.List<java.lang.String> getMetadataUIDs()

Retrieves the list of UIDs of all metadata records. Useful for stream().forEach(...) operations.

Returns:

The list of all metadata UIDs.

getNumberOfMolecules

int getNumberOfMolecules()

Number of molecule records in the MoleculeArchive.

Returns:

The integer number of molecule records.

getNumberOfMetadatas

int getNumberOfMetadatas()

Number of metadata records in the MoleculeArchive.

Returns:

The integer number of metadata records.

getComments

java.lang.String getComments()

Global comments.

Returns:

The global comments String.

setComments

void setComments(java.lang.String comments)

Sets the global comments. This replaces all current comments with those given.

Parameters:

comments - A string of global comments to set.

isVirtual

boolean isVirtual()

True if the archive is virtual, false if not.

Returns:

A boolean which is true if working from a virtual store.

remove

void remove(java.lang.String UID)

Removes the molecule record with the given UID.

Parameters:

UID - The UID of the molecule record to remove.

remove

void remove(M molecule)

Removes the molecule record provided.

Parameters:

molecule - The molecule record to remove.

getMoleculeUIDs

java.util.List<java.lang.String> getMoleculeUIDs()

Retrieves the list of UIDs for all Molecule records. Useful for stream().forEach(...) operations.

Returns:

The list with all Molecule UIDs.

getTagList

java.lang.String getTagList(java.lang.String UID)

Comma separated list of tags for the molecule with the given UID.

Parameters:

UID - The UID of the molecule to retrieve the tag list for.

Returns:

A String containing a comma separated list of tags.

getTagSet

java.util.Set<java.lang.String> getTagSet(java.lang.String UID)

Tags for the molecule with the given UID.

Parameters:

UID - The UID of the molecule to retrieve the tag set for.

Returns:

A set containing all tags for the given molecule.

getChannel

int getChannel(java.lang.String UID)

Channel for the molecule with the given UID.

Parameters:

UID - The UID of the molecule to retrieve the channel of.

Returns:

The channel index of the molecule in question.

getImage

int getImage(java.lang.String UID)

Image index for the molecule with the given UID.

Parameters:

UID - The UID of the molecule to retrieve the image index of.

Returns:

The image index of the molecule in question.

getMetadataTagList

java.lang.String getMetadataTagList(java.lang.String UID)

Comma separated list of tags for the metadata record with the given UID.

Parameters:

UID - The UID of the metadata record to retrieve the tag list for.

Returns:

A String containing a comma separated list of tags.

getMetadataTagSet

java.util.Set<java.lang.String> getMetadataTagSet(java.lang.String UID)

Tags for the metadata record with the given UID.

Parameters:

UID - The UID of the metadata record to retrieve the tag list for.

Returns:

The set of tags for the given metadata record.

getMoleculeBatch

java.util.List<double[][]> getMoleculeBatch(java.util.List<java.lang.String> UIDs,
                                            java.lang.String tColumn,
                                            java.lang.String signalColumn,
                                            int rangeStart,
                                            int rangeEnd,
                                            java.util.List<java.lang.String> tagsToLearn,
                                            int threads)

Utility function to generate batches of molecules data in an optimal format for machine learning using keras. Region goes from rangeStart to 1 - rangeEnd.

Parameters:

UIDs - The list of UIDs for the molecule to review data from.

tColumn - Name of the T column.

signalColumn - Name of the signal column.

rangeStart - Index of start of range in T column.

rangeEnd - Index of end of range in T column.

tagsToLearn - List of tags to use to build labels.

threads - Number of thread to use when building data.

Returns:

Returns batch of molecule data.

moleculeHasTag

boolean moleculeHasTag(java.lang.String UID,
                       java.lang.String tag)

Check if a molecule record has a tag. This offers optimal performance for virtual mode because only the tag index is checked without retrieving all virtual records.

Parameters:

UID - The UID of the molecule to check for the tag.

tag - The tag to check for.

Returns:

Returns true if the molecule has the tag and false if not.

moleculeHasTags

boolean moleculeHasTags(java.lang.String UID)

Check if a molecule record has tags. This offers optimal performance for virtual mode because only the tag index is checked without retrieving all virtual records.

Parameters:

UID - The UID of the molecule to check.

Returns:

Returns true if the molecule has tags and false if not.

moleculeHasNoTags

boolean moleculeHasNoTags(java.lang.String UID)

Check if a molecule record has no tags. This offers optimal performance for virtual mode because only the tag index is checked without retrieving all virtual records.

Parameters:

UID - The UID of the molecule to check.

Returns:

Returns true if the molecule has no tags and false if it has tags.

addMoleculeTags

void addMoleculeTags(java.util.Map<java.lang.String,java.lang.String> tagMap)

Add tags to molecules using UID to tag map. This offers optimal performance by using multiple threads. Provides a way to add tags resulting from machine learning using python.

Parameters:

tagMap - The UID to tag map for add to molecules.

moleculeTags

java.util.Set<java.lang.String> moleculeTags(java.lang.String UID)

Retrieve the list of tags for a molecule. Will retrieve the list from the index if working in virtual memory.

Parameters:

UID - The UID of the molecule to retrieve the tags of.

Returns:

Returns the set of for the molecule with UID.

metadataHasTag

boolean metadataHasTag(java.lang.String UID,
                       java.lang.String tag)

Check if a MARSImageMetadata record has a tag. This offers optimal performance for virtual mode because only the tag index is checked without retrieving all virtual records.

Parameters:

UID - The UID of the MARSImageMetadata record to check for the tag.

tag - The tag to check for.

Returns:

Returns true if the MARSImageMetadata record has the tag and false if not.

deleteMoleculesWithTag

void deleteMoleculesWithTag(java.lang.String tag)

Removes all molecule records with the tag provided.

Parameters:

tag - Molecule records with this tag will be removed.

deleteMetadatasWithTag

void deleteMetadatasWithTag(java.lang.String tag)

Removes all metadata records with the tag provided.

Parameters:

tag - metadata records with this tag will be removed.

contains

boolean contains(java.lang.String UID)

Used to check if there is a molecule record with the UID given.

Parameters:

UID - Check for a molecule record with this UID.

Returns:

True if the archive contains the molecule record with the provided UID and false if not.

containsMetadata

boolean containsMetadata(java.lang.String UID)

Used to check if there is a metadata record with the UID given.

Parameters:

UID - Check for a metadata record with this UID.

Returns:

True if the archive contains a metadata record with the provided UID and false if not.

get

M get(java.lang.String UID)

Get the molecule record with the given UID.

Parameters:

UID - The UID of the record to retrieve.

Returns:

The Molecule record with the UID given or null if none is located.

get

M get(int index)

Retrieves the molecule record at the provided index.

Parameters:

index - The integer index position of the molecule record.

Returns:

A Molecule record.

molecules

java.util.stream.Stream<M> molecules()

Convenience method to retrieve a Molecule stream. Can be used to iterate over all molecules using forEach.

Returns:

Molecule stream.

metadata

java.util.stream.Stream<I> metadata()

Convenience method to retrieve a metadata stream. Can be used to iterate over all metadata using forEach.

Returns:

Metadata stream.

parallelMetadata

java.util.stream.Stream<I> parallelMetadata()

Convenience method to retrieve a multithreated Molecule stream. Can be used to iterate over all molecules using forEach in a multithreaded manner.

Returns:

Molecule stream.

parallelMolecules

java.util.stream.Stream<M> parallelMolecules()

Convenience method to retrieve a multithreaded Molecule stream. Can be used to iterate over all molecules using forEach in a multithreaded manner.

Returns:

Molecule stream.

getMetadataUIDforMolecule

java.lang.String getMetadataUIDforMolecule(java.lang.String UID)

Get the UID of the metadata for a molecule record. If working from a virtual store, this will use an index providing optimal performance. If working in memory this is the same as retrieving the molecule record and the metadata UID from it directly.

Parameters:

UID - The UID of the molecule to get the metadata UID for.

Returns:

The UID string of the metadata record corresponding to the molecule record whose UID was provided.

getFile

java.io.File getFile()

Returns the File from which the archive was opened.

Returns:

The File the archive was opened from.

setFile

void setFile(java.io.File file)

Set the file the archive should save to.

Parameters:

file - The File the archive was opened from.

setName

void setName(java.lang.String name)

Set the name of the archive.

Parameters:

name - The new name of the archive.

getName

java.lang.String getName()

Get the name of the archive.

Returns:

The String name of the archive.

getWindow

MoleculeArchiveWindow getWindow()

Returns the MoleculeArchiveWindow holding this archive, if one exists. Otherwise, null is returned.

Returns:

The MoleculeArchiveWindow containing this archive.

setWindow

void setWindow(MoleculeArchiveWindow win)

Set the window containing this archive.

Parameters:

win - Set the MoleculeArchiveWindow that contains this archive.

log

void log(java.lang.String str)

Add a log message to all MarsImageMetadata records. Used by all processing plugins so there is a record of the sequence of processing steps during analysis. Do not start a new line after adding the message.

Parameters:

str - The String message to add to all MarsMetadata logs.

logln

void logln(java.lang.String str)

Add a log message to all MarsImageMetadata records. Used by all processing plugins so there is a record of the sequence of processing steps during analysis. Start a new line after adding the message.

Parameters:

str - The String message to add to all MarsImageMetadata logs.

createIndex

X createIndex()

Create empty MoleculeArchiveIndex.

Returns:

An empty MoleculeArchiveIndex.

createIndex

X createIndex(com.fasterxml.jackson.core.JsonParser jParser)
       throws java.io.IOException

Create empty MoleculeArchiveIndex using the JsonParser stream given.

Parameters:

jParser - JsonParser to use to create the MoleculeArchiveIndex.

Returns:

MoleculeArchiveIndex created using the JsonParser stream.

Throws:

java.io.IOException - Thrown if unable to read Json from JsonParser stream.

properties

MoleculeArchiveProperties<M,I> properties()

Get the MoleculeArchiveProperties which contain general information about the archive. This includes numbers of records, comments, and global lists of table columns, tags, and parameters.

Returns:

The MoleculeArchiveProperties for the MoleculeArchive.

createProperties

P createProperties()

Create empty MoleculeArchiveProperties record.

Returns:

Empty MoleculeArchiveProperties.

createProperties

P createProperties(com.fasterxml.jackson.core.JsonParser jParser)
            throws java.io.IOException

Create MoleculeArchiveProperties record using JsonParser stream.

Parameters:

jParser - JsonParser to use to create archive properties.

Returns:

MoleculeArchiveProperties created.

Throws:

java.io.IOException - Thrown if unable to read Json from JsonParser stream.

createMetadata

I createMetadata(com.fasterxml.jackson.core.JsonParser jParser)
          throws java.io.IOException

Create MarsMetadata record using JsonParser stream.

Parameters:

jParser - JsonParser to use to create metadata.

Returns:

MarsMetadata record created using JsonParser stream.

Throws:

java.io.IOException - Thrown if unable to read Json from JsonParser stream.

createMetadata

I createMetadata(java.lang.String metaUID)

Create empty MarsMetadata record with the metaUID specified.

Parameters:

metaUID - The metaUID to use during creation of the empty MarsMetadata record.

Returns:

MarsMetadata record.

createMolecule

M createMolecule()

Create empty Molecule record.

Returns:

Empty molecule record.

createMolecule

M createMolecule(com.fasterxml.jackson.core.JsonParser jParser)
          throws java.io.IOException

Create Molecule record using the JsonParser stream given.

Parameters:

jParser - JsonParser to use to create the molecule.

Returns:

Molecule created using the JsonParser stream.

Throws:

java.io.IOException - Thrown if unable to read Json from JsonParser stream.

createMolecule

M createMolecule(java.lang.String UID)

Create empty Molecule record with the UID specified.

Parameters:

UID - The UID to use during creation.

Returns:

Empty molecule with the UID given.

createMolecule

M createMolecule(java.lang.String UID,
                 MarsTable table)

Create Molecule record using the UID and MarsTable specified.

Parameters:

UID - The UID to use during molecule creation.

table - The MarsTable set as the DataTable during creation.

Returns:

Molecule created.