In my personal time, I occasionally take part in bug bounty programs. The idea is to find vulnerabilities in open source libraries and report them to the maintainers before any malicious hackers find them. You get a financial reward — it doesn't pay much, it's very time-consuming, but it's rather fun.
I found a vulnerability in the h5py package. It makes it possible to embed, inside
HDF5 or netCDF4 files, a program that will be executed by
the machine reading those files.
This could, for example, allow a hacker to spread a virus through this file format.
Someone (or an LLM) beat me to it and pocketed the bug bounty reward before me. It would have bothered me to have spent so much time on this topic without it leading anywhere, so I decided to turn it into a LinkedIn post.
I tried to make the subject as accessible as possible — it's not an exercise I'm used to, and the result is still fairly technical. You've been warned :)
To set the context: h5py is the de facto standard for reading and writing
HDF5 files in Python. HDF5 is a popular format in data science,
oceanography, machine learning, and every other discipline where you need to store
large amounts of data in an easily indexable format.
h5py doesn't manipulate HDF5 files directly; it's a wrapper around the HDF5 library, written in C by
the HDF Group. The vast majority of the parsing/reading/writing work is done by
that C library. h5py is "just" a layer on top, letting you work with files
from Python in an ergonomic way, without sacrificing performance.
The vulnerability lies in this wrapper, not in the underlying library.
For the sake of ergonomics, h5py lets you store numpy objects directly in files.
It implements a "translation" layer between in-memory numpy objects and files.
The translation happens automatically when h5py reads the file.
It's very convenient — almost every data science program written in Python uses numpy.
For example, if a user writes a numpy.datetime64 to a file,
h5py will translate it into a uint64.
It annotates the associated dataset with instructions that allow the reverse
transformation when the file is read.
It looks like this:
# /// script
# requires-python = ">=3.14"
# dependencies = [
# "h5py",
# "numpy",
# ]
# ///
import numpy as np
import h5py
with h5py.File("dates.h5", "w") as dataset:
# create an array of timestamps,
# of type numpy.datetime64
data = np.array([np.datetime64('2000-01-01T00:00:00')])
# store it in the file
dataset['data'] = data.astype(h5py.opaque_dtype(data.dtype))
If we then look at the contents of the dates.h5 file with h5dump:
$ h5dump dates.h5
HDF5 "dates.h5" {
GROUP "/" {
DATASET "data" {
DATATYPE H5T_OPAQUE {
OPAQUE_TAG "NUMPY:<M8[s]"; # <-- type description (a)
}
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
DATA {
(0): 80:43:6d:38:00:00:00:00 # <-- 8 bytes of data (b)
}
}
}
}
We can see that h5py defined a DATASET, with a single element, of type H5T_OPAQUE.
The type itself is annotated with a tag: NUMPY:<M8[s]. This is a string
that h5py added so it can reconstruct the date from the stored data,
here: '80:43:6d:38:00:00:00:00'.
By reading h5py's code, we understand that the part following NUMPY: (here <M8[s])
is used directly to interpret the data type when reading the file,
without any validation.
When the library reads this data, it does the equivalent of this:
date = np.frombuffer(
0x80436d3800000000.to_bytes(8), # the 8 bytes of data (b)
dtype="<M8[s]", # the type (a)
)
assert date == np.datetime64('2000-01-01T00:00:00')
The thing is, you can write whatever you want in that tag,
as long as it respects the type description format that numpy understands (https://numpy.org/doc/2.1/reference/arrays.dtypes.html#specifying-and-constructing-data-types).
h5py will try to interpret the file's data as whatever type you specify.
Here's where it gets interesting: numpy defines an O type for "objects".
It allows storing references to Python objects inside numpy arrays.
This is useful if you want to put objects numpy doesn't natively support into an array.
For example, you can easily make an array of dictionaries:
# /// script
# requires-python = ">=3.14"
# dependencies = [
# "numpy",
# ]
# ///
import numpy as np
a = {}
c = np.array([a])
assert c[0] is a
Here, c doesn't contain a itself, but the address of the piece of memory that holds it.
One could write an equivalent program using the O format explicitly, along with the address of a,
but numpy doesn't allow it:
import numpy as np
a = {}
addr_of_a = id(a) # CPython uses the memory address of objects to identify them.
# here it crashes, numpy refuses!
c = np.frombuffer(
addr_of_a.to_bytes(8),
dtype="O",
)
Why does numpy refuse to create an array of references from a raw buffer? Because it has no guarantee that the pointed-to memory contains a valid object, or even that it is allocated at all.
In C, this would amount to doing PyObject* c = (PyObject *) addr_of_a;.
If addr_of_a doesn't point to a memory segment allocated to the process,
dereferencing c triggers a segfault.
However, through h5py, it becomes possible to do the equivalent. All it takes is writing
an arbitrary address into a file, and annotating its type as "NUMPY:|O"
import numpy as np
import h5py
a = {}
addr_of_a = id(a) # (under CPython, id() returns the object's address)
# write
with h5py.File("data.h5", "w") as dataset:
# we create an 8-byte "opaque" HDF5 type,
# and annotate it with the numpy format that describes addresses
dtype = h5py.h5t.create(h5py.h5t.OPAQUE, 8)
dtype.set_tag(b"NUMPY:|O")
# we write the address of `a` into the file
space = h5py.h5s.create_simple((1,))
ds = h5py.h5d.create(dataset.id, b"a", dtype, space)
ds.write(space, space, np.array([addr_of_a]), dtype)
# read
with h5py.File("data.h5", "r") as dataset:
b = dataset["a"][0]
assert b is a # by re-reading the file we just wrote,
# we end up back on `a`
Here the script does 3 things:
aobjecthdf5 parses its contents to create b, which points to a(The write part is more complex than before; here I'm using h5py's low-level
API to bypass a validation step that prevents writing "NUMPY:|O", but the idea is the same).
You can probably see where I'm going: what would happen if a didn't exist in the process?
If we take the same data.h5 file and read it in a new script:
import h5py
with h5py.File("data.h5", "r") as dataset:
a = dataset["a"][0]
Then the script doesn't 'print' anything at all. However, if we look at the kernel logs at the same time, we notice it generated a segfault:
yves@laptop $ dmesg
...
python3[674234]: segfault at 745a86699040 ...
Python tried to dereference the address 745a86699040, which here points to nothing at all.
Actually, it's worse than that: this address isn't even included in the memory space allocated to the process; the kernel detected it
and killed the program.
That's already enough to crash the program of any user who would read this file — in itself, that's already a vulnerability. I wanted to push the research further, all the way to arbitrary code execution, but I could already have pocketed a small bounty by reporting this vulnerability. In hindsight, I realize that's what I should have done, since someone else got ahead of me, collected the bounty, and went no further.
If you've followed this far (congrats!), you might be wondering why the script causes a segfault?
a = dataset["a"][0] binds the variable a to the object whose address is stored in the file, but
this variable is never read afterwards — so why does Python try to dereference it?
It's the Python runtime that's responsible. To avoid memory leaks, CPython deallocates every object as soon as the program no longer needs it. To detect when an object becomes "useless", it uses a very simple heuristic: if no variable points to an object, then the object is unreachable and can be deleted.
For this strategy to work, the number of references pointing to each object must be stored somewhere.
This counter is updated every time a variable is bound to or unbound from the object.
CPython calls this counter a refcount, and it stores it directly inside every object.
It's even the first field of the PyObject structure, the one that represents a generic Python object.
In other words, as soon as the variable a goes out of scope, CPython decrements the first uint64 in the piece of memory
pointed to by a.
In the previous script, this led to a segfault because the refcount CPython tried to decrement
was not in a memory zone allocated to the process. On the other hand, if it had been, the address would indeed have been
decremented (that's exactly what happened in the example that did the write and then the read).
So we can craft a corrupted file capable of modifying the memory of the process that reads it. And that is enough to execute arbitrary code.
The idea is as follows: decrement a function pointer to redirect its execution to a piece of code we would have carefully chosen.
To do this, we must find, in cpython's memory space, a function pointer that:
I had a very hard time finding pointers satisfying all these conditions, but I eventually found the holy grail.
For some time now (CPython 3.12?), the PyTypeObject objects
that describe the most common primitive types (list, tuple, set, frozenset, ...) are placed in a static
segment of the CPython process (therefore not randomized).
These objects thus have an address you can predict in advance (provided you know which Python version the victim is using).
We can then "decrement" these addresses at will to modify their default methods, such as tp_dealloc, which points to the C function
to call to deallocate these objects.
That's the path I followed. My "proof of concept" builds an HDF5 dataset which, when read:
tp_dealloc of several default types to replace them with "ROP gadgets" (addresses pointing to portions of code in CPython's text segment),tp_dealloc functions, in an order specially chosen so that they pivot the stack
toward a buffer I control,ROP chain via this new stack, to finally run /proc/self/exe -c "my script"I'm glossing over the selection of the gadgets that make this "exploit" work, but that was by far the most time-consuming part.
You have to disassemble cpython to search it for portions of functions (in asm) that, stitched together, execute the target code.
It's the only way I found to defeat Address Space Layout Randomization.
It was very tedious, but it let me brush up on my x86_64 :D
A demo of the exploit is available on my GitHub