62 lines
3.4 KiB
Go
62 lines
3.4 KiB
Go
|
// Copyright (c) 2026 Petr Balvín <opensource@petrbalvin.org> (https://petrbalvin.org)
|
||
|
|
// SPDX-License-Identifier: MIT
|
||
|
|
|
||
|
|
// Package io reads and writes the formats scientific data arrives in:
|
||
|
|
// comma-separated text, FITS images and tables, HDF5, NetCDF classic
|
||
|
|
// and native-endian memory maps. Every loader returns the library's
|
||
|
|
// own Array, so a file read is an ordinary value the rest of the
|
||
|
|
// library operates on, and every writer takes one.
|
||
|
|
//
|
||
|
|
// CSV is the interchange format for tabular data with spreadsheets
|
||
|
|
// and statistical packages: LoadCSV, LoadCSVReader, SaveCSV and
|
||
|
|
// SaveCSVWriter handle 2-D arrays, with or without a header row. The
|
||
|
|
// writer formats every numeric dtype the core carries: the integer
|
||
|
|
// class as exact integer text, booleans as 0 and 1, floats as float
|
||
|
|
// text; complex is refused, because CSV carries plain numeric text
|
||
|
|
// and a pair of raw halves would read back as two unrelated columns.
|
||
|
|
// The readers parse everything back as float64.
|
||
|
|
//
|
||
|
|
// FITS is astronomy's archival format. LoadFITS reads a primary image
|
||
|
|
// (BITPIX -64 and -32) with its header cards and applies the
|
||
|
|
// BSCALE/BZERO scaling; SaveFITS writes one. LoadFITSTable and
|
||
|
|
// SaveFITSTable cover the binary and ASCII table extensions, where
|
||
|
|
// catalogues and observation logs live.
|
||
|
|
//
|
||
|
|
// HDF5 is read by LoadHDF5, which returns every dataset of a file by
|
||
|
|
// path, with the attributes of the groups it sits in merged into it.
|
||
|
|
// Superblocks 0 to 3, object headers of version 1 and 2, symbol-table
|
||
|
|
// and link-message groups, contiguous, compact and chunked storage and
|
||
|
|
// the deflate, shuffle and fletcher32 filters are supported. Dataset
|
||
|
|
// values land the core dtype their datatype declares: fixed-point data
|
||
|
|
// by stored width and signedness, the boolean enumeration convention
|
||
|
|
// as Bool, floating-point data as float64 or float32 by width. What is
|
||
|
|
// not supported, among it dense groups, the version 2 chunk B-tree,
|
||
|
|
// string datasets, every big-endian datatype, bit fields, non-boolean
|
||
|
|
// enumerations and unsigned 64-bit integers, is refused with an error
|
||
|
|
// naming it. SaveHDF5 writes the mirror image in the classic layout
|
||
|
|
// or, with Latest, the superblock 3 layout, and SaveHDF5Text writes
|
||
|
|
// fixed-length string datasets.
|
||
|
|
//
|
||
|
|
// NetCDF classic (CDF-1 and CDF-2) is the archival format of climate
|
||
|
|
// and ocean science: LoadNetCDF returns named dimensions, variables
|
||
|
|
// and global attributes, and SaveNetCDF writes CDF-1. Each variable
|
||
|
|
// lands the core dtype its classic type code carries: NC_BYTE as
|
||
|
|
// int8, NC_CHAR as uint8 raw bytes (CHAR carries bytes at the array
|
||
|
|
// level, never text), NC_SHORT as int16, NC_INT as int32, and
|
||
|
|
// NC_FLOAT and NC_DOUBLE as float64. A variable that lands a narrow
|
||
|
|
// dtype from a file is refused by SaveNetCDF, whose writer stores
|
||
|
|
// float64, float32 and int64 arrays only; convert with Astype first.
|
||
|
|
// Record dimensions are read and written in both directions; the
|
||
|
|
// writer stores float64, float32 and int64 arrays, and a type code
|
||
|
|
// beyond the classic six is refused by name.
|
||
|
|
//
|
||
|
|
// MapFloats, MapFloat32s and MapInts open a native-endian file as a
|
||
|
|
// read-only array without reading it, which suits data far larger than
|
||
|
|
// memory; release unmaps it and comes strictly last. SaveNativeFloats
|
||
|
|
// writes the format MapFloats reads.
|
||
|
|
//
|
||
|
|
// Each format is covered by a documented subset and the rest is
|
||
|
|
// refused by name, never half-read. Errors carry the library's
|
||
|
|
// "tensor: " prefix and the name of the call that produced them.
|
||
|
|
package io
|