How to memory-map a NumPy array

Memory mapping lets NumPy work with array data that lives in a disk file instead of forcing a full array read into RAM. It fits large NPY datasets where a script needs to fill, inspect, or update selected slices while keeping the file usable by later NumPy runs.

For NPY files, numpy.lib.format.open_memmap() creates or opens a memory-mapped array while keeping the file header compatible with np.load(). np.load(path, mmap_mode=“r”) reopens an existing NPY file as a read-only memmap for inspection.

Writes through an r+ or w+ mapping should be flushed before another process or a later load needs the changed bytes. Keep one writer responsible for a mapped file, and use read-only mappings for readers that only need slices.

1. Create a script that builds an NPY-backed memory map with a fixed shape and dtype.

   array-memory-map.py

   import numpy as np
   from numpy.lib.format import open_memmap
   from pathlib import Path
    
   path = Path("sensor-readings.npy")
    
   mapped = open_memmap(path, mode="w+", dtype=np.float32, shape=(3, 4))
   mapped[:] = np.arange(12, dtype=np.float32).reshape(3, 4)
   mapped[-1, -1] = 99
   mapped.flush()
    
   readonly = np.load(path, mmap_mode="r")
    
   print("mapped type:", type(mapped).__name__)
   print("readonly type:", type(readonly).__name__)
   print("mapped shape:", mapped.shape)
   print("mapped dtype:", mapped.dtype)
   print("readonly writeable:", readonly.flags.writeable)
   print("last row:", readonly[-1].tolist())
   print("saved value:", float(np.load(path)[-1, -1]))

   open_memmap() creates a .npy file and returns a memmap object. For an existing NPY file, use np.load(path, mmap_mode=“r+”) when the script must save edits, or mmap_mode=“r” when it only reads slices.

2. Run the script.

   $ python array-memory-map.py
   mapped type: memmap
   readonly type: memmap
   mapped shape: (3, 4)
   mapped dtype: float32
   readonly writeable: False
   last row: [8.0, 9.0, 10.0, 99.0]
   saved value: 99.0

   readonly writeable: False confirms the second mapping cannot modify the file. The saved value comes from a normal reload after mapped.flush().

3. Remove the sample NPY file.

   $ rm sensor-readings.npy

   Keep the file instead when it is the dataset another process or later run should reuse. NumPy does not expose a dedicated close method for memmap, so let mappings go out of scope before replacing or deleting the file.