| Title: | Convert Compressed Mobile Voice Notes into Tidy Spectrogram Data |
|---|---|
| Description: | Reads compressed '.opus'/'.ogg' voice notes -- the kind produced by WhatsApp and similar VoIP messaging apps -- and converts them directly into tidy data frames of spectrogram and spectral-feature data, ready for analysis or modelling. Also provides a function to simulate the lossy compression such messaging apps apply, for testing how well a signal survives it. Built for a bioacoustic pest-detection pipeline (SonicScout) but has no dependency on that domain. |
| Authors: | Anirjit Deb Adhikary [aut, cre] |
| Maintainer: | Anirjit Deb Adhikary <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.0 |
| Built: | 2026-11-01 16:21:56 UTC |
| Source: | https://github.com/jitu-droid/sonicscout |
Reads every supported audio file under dir (recursively) and applies
fn (default wa_features()) to each, row-binding the results with a
leading file column identifying the source path.
wa_batch(dir, fn = wa_features, pattern = "\\.(opus|ogg|wav|mp3|m4a)$", ...)wa_batch(dir, fn = wa_features, pattern = "\\.(opus|ogg|wav|mp3|m4a)$", ...)
dir |
Directory to scan for audio files. |
fn |
Function taking a tuneR::Wave object and returning a
single-row data frame/tibble. Defaults to |
pattern |
Regex for filenames to include. Defaults to the
extensions |
... |
Passed on to |
A tibble: one row per file, with a leading file column, plus
whatever columns fn returns.
## Not run: features_df <- wa_batch("voice_notes/", wa_features) ## End(Not run)## Not run: features_df <- wa_batch("voice_notes/", wa_features) ## End(Not run)
Extracts a documented, inspectable feature set rather than a black-box
embedding, so results can be explained rather than just trusted:
peak_freq_hz, spectral_centroid_hz, spectral_flatness,
spectral_entropy, spectral_rolloff85_hz (frequency below which 85% of
spectral energy lies), dominant_freq_sd_hz (stability of the dominant
frequency contour over time – low means tonal/steady, high means
erratic), harmonic_energy_ratio (a simple heuristic, see Details), and
12 MFCCs (mean over frames).
wa_features(wave, wl = 1024)wa_features(wave, wl = 1024)
wave |
A tuneR::Wave object, e.g. from |
wl |
Window length (samples) used for the underlying STFT/mean
spectrum. See |
harmonic_energy_ratio is the fraction of total spectral energy that
falls within a narrow band around the first 5 integer multiples of the
peak frequency. It is a lightweight, transparent heuristic for "how
harmonic does this sound look", not a validated harmonicity/HNR index –
report it as such.
A one-row tibble.
## Not run: w <- wa_read("voice_note.opus") wa_features(w) ## End(Not run)## Not run: w <- wa_read("voice_note.opus") wa_features(w) ## End(Not run)
Handles the container/codec mess of mobile voice notes: .opus and
.ogg files are decoded via a system ffmpeg binary into a temporary
WAV before being read with tuneR::readWave(). .wav files are read
directly, and .mp3/.m4a are routed through the same decode step as
opus/ogg.
wa_read(path, sample_rate = NULL, mono = FALSE)wa_read(path, sample_rate = NULL, mono = FALSE)
path |
Path to an audio file ( |
sample_rate |
Optional target sample rate in Hz. |
mono |
If |
A tuneR::Wave object.
av
The obvious implementation would use av::av_audio_convert() (av binds
FFmpeg's libav* libraries directly, no subprocess needed). That was tried
first, but on the machine this package was built and tested on,
av::av_audio_convert() and even av::av_media_info() segfault the R
process outright – confirmed even against av's own bundled sample
file, so it isn't an input-file issue. A segfault can't be caught with
tryCatch(), which makes av unsafe to call from a long-running
process (a webhook server, a Shiny session) on that machine. See
BLOCKERS.md in the parent repo for the full diagnosis. This function
shells out to a system ffmpeg binary instead, which is confirmed
working. If a future av version/rebuild is confirmed not to segfault
on your machine, switching back to av::av_audio_convert() would avoid
the subprocess-spawn overhead.
By default the audio comes back at whatever rate the file decodes to –
48 kHz for Opus, usually 44.1 or 48 kHz for MP3/WAV. A model trained at
one sample rate must be fed that same rate: spectral features computed
over a different bandwidth are not comparable. Pass sample_rate (and
mono = TRUE) to have ffmpeg resample on decode, using its filtered
resampler (with anti-aliasing), not naive decimation.
## Not run: w <- wa_read("voice_note.opus") w16 <- wa_read("voice_note.opus", sample_rate = 16000, mono = TRUE) ## End(Not run)## Not run: w <- wa_read("voice_note.opus") w16 <- wa_read("voice_note.opus", sample_rate = 16000, mono = TRUE) ## End(Not run)
Round-trips a tuneR::Wave object through the same codec pipeline
WhatsApp (and similar apps) apply to voice notes: resample to sr Hz,
downmix to mono, encode to Opus at bitrate bits/sec inside an Ogg
container, then decode back to PCM.
wa_simulate_compression(wave, bitrate = 16000, sr = 16000)wa_simulate_compression(wave, bitrate = 16000, sr = 16000)
wave |
A tuneR::Wave object. |
bitrate |
Target bitrate in bits/sec (default |
sr |
Target sample rate in Hz (default |
A tuneR::Wave object: the degraded, round-tripped audio.
av::av_audio_convert() would be the obvious choice here, and its
installed version does have a bit_rate argument – but empirically it
(and even av::av_media_info()) segfaults R outright on the machine this
package was built and tested on, including against av's own bundled
sample file. A segfault can't be caught with tryCatch(), which makes it
unsafe to call from a long-running process. See BLOCKERS.md in the
parent repo. Both the encode and decode steps here shell out to a system
ffmpeg binary (with libopus) instead. This requires ffmpeg to be on
PATH; call Sys.which("ffmpeg") to check.
## Not run: w <- wa_read("clean_recording.wav") degraded <- wa_simulate_compression(w, bitrate = 16000, sr = 16000) ## End(Not run)## Not run: w <- wa_read("clean_recording.wav") degraded <- wa_simulate_compression(w, bitrate = 16000, sr = 16000) ## End(Not run)
Wraps seewave::spectro() and reshapes its matrix output into a long
tibble of time / freq / amplitude, which composes far more easily
with dplyr/ggplot2 than the raw list-of-matrices seewave returns.
wa_spectrogram(wave, wl = 1024, ovlp = 50)wa_spectrogram(wave, wl = 1024, ovlp = 50)
wave |
A tuneR::Wave object, e.g. from |
wl |
Window length (samples) for the STFT. Longer windows give better frequency resolution at the cost of time resolution – for low-frequency signals (like insect wingbeats) prefer a longer window (e.g. 1024-2048) over the seewave default. |
ovlp |
Percent overlap between successive analysis windows. |
A tibble with columns time (seconds), freq (kHz, matching
seewave's convention), and amplitude.
## Not run: w <- wa_read("voice_note.opus") spec_df <- wa_spectrogram(w, wl = 2048) ## End(Not run)## Not run: w <- wa_read("voice_note.opus") spec_df <- wa_spectrogram(w, wl = 2048) ## End(Not run)