ndsampler.frame_cache

Tools for caching intermediate frame representations.

Module Contents

Functions

_cog_cache_write(gpath, cache_gpath[, config])

CommandLine:

_npy_cache_write(gpath, cache_gpath[, config])

_locked_cache_write(_write_func, gpath, cache_gpath[, ...])

Ensures that mem_gpath exists in a multiprocessing-safe way

_lookup_dvc_hash(path)

proof of concept

_ensure_image_npy(gpath, cache_gpath)

Ensures that cache_gpath exists in a multiprocessing-safe way

_ensure_image_cog(gpath, cache_gpath, config[, ...])

Returns a special array-like object with a COG GeoTIFF backend

Attributes

DEBUG_COG_ATOMIC_WRITE

DEBUG_FILE_LOCK_CACHE_WRITE

RUN_COG_CORRUPTION_CHECKS

DEBUG_LOAD_COG

ndsampler.frame_cache.DEBUG_COG_ATOMIC_WRITE = 0
ndsampler.frame_cache.DEBUG_FILE_LOCK_CACHE_WRITE = 0
ndsampler.frame_cache.RUN_COG_CORRUPTION_CHECKS = True
ndsampler.frame_cache.DEBUG_LOAD_COG
exception ndsampler.frame_cache.CorruptCOG[source]

Bases: Exception

Common base class for all non-exit exceptions.

ndsampler.frame_cache._cog_cache_write(gpath, cache_gpath, config=None)[source]
CommandLine:

xdoctest -m ndsampler.abstract_frames _cog_cache_write

Example

>>> # xdoctest: +REQUIRES(module:osgeo)
>>> import ndsampler
>>> from ndsampler.abstract_frames import *
>>> import kwcoco
>>> workdir = ub.ensure_app_cache_dir('ndsampler')
>>> dset = kwcoco.CocoDataset.demo()
>>> imgs = dset.images()
>>> id_to_name = imgs.lookup('file_name', keepid=True)
>>> id_to_path = {gid: join(dset.img_root, name)
>>>               for gid, name in id_to_name.items()}
>>> self = SimpleFrames(id_to_path, workdir=workdir)
>>> image_id = ub.peek(id_to_name)
>>> #gpath = self._lookup_gpath(image_id)

#### EXIT # >>> hashid = self._lookup_hashid(image_id) # >>> cog_gname = ‘{}_{}.cog.tiff’.format(image_id, hashid) # >>> cache_gpath = cog_gpath = join(self.cache_dpath, cog_gname) # >>> _cog_cache_write(gpath, cache_gpath, {})

ndsampler.frame_cache._npy_cache_write(gpath, cache_gpath, config=None)[source]
ndsampler.frame_cache._locked_cache_write(_write_func, gpath, cache_gpath, config=None)[source]

Ensures that mem_gpath exists in a multiprocessing-safe way

ndsampler.frame_cache._lookup_dvc_hash(path)[source]

proof of concept

Ignore:

path = ‘/home/joncrall/data/dvc-repos/viame_dvc/public/Benthic/US_NE_2017_CFF_HABCAM/annotations_flatfish.kwcoco.json’ _lookup_dvc_hash(path) path = ‘/home/joncrall/data/dvc-repos/viame_dvc/public/Benthic/US_NE_2017_CFF_HABCAM/Left/201509.20170718.222415535.269652.png’ _lookup_dvc_hash(path)

ndsampler.frame_cache._ensure_image_npy(gpath, cache_gpath)[source]

Ensures that cache_gpath exists in a multiprocessing-safe way

Returns a memmapped reference to the entire image

ndsampler.frame_cache._ensure_image_cog(gpath, cache_gpath, config, hack_use_cli=True)[source]

Returns a special array-like object with a COG GeoTIFF backend