From Soundtrack to Sonic Ecosystem

Abstract

Core Resonance proposes that a musical world be an active instrument: player, AI companion, Nodes, materials and creatures both shape and respond to sound. This concept paper locates that proposal within digital musical instrument mapping and virtual acoustics research, then specifies a modest, falsifiable prototype. Existing scholarship verifies the relevance of input-to-sound mapping and modeled acoustic propagation; it does not verify Core Resonance implementation, physical accuracy, or user benefits.

1. Research problem

A conventional game places tracks alongside gameplay; the Core Resonance brief instead specifies Explore - Discover - Listen - Connect - Create - Adapt - Evolve and a loop in which musical action changes the environment, whose response in turn changes music. The design question is whether players can perceive, predict, and deliberately influence these bidirectional mappings. A beautiful animation is not equivalent to controllable musical feedback.

The current project documentation distinguishes Instrument Nodes, Agent Nodes, organisms, geometry, materials, player input and JamBot. The long-term ambition is volumetric, ecosystem-scale resonance. The 3D implementation starter explicitly rejects building a physically complete wave solver first; it proposes a sparse spatial event and separate visual shell. Those are design specifications, not validated runtime features.

2. Established scholarly foundations

Hunt, Wanderley and Paradis [1] discuss the central importance of parameter mapping between gesture and synthesis in digital musical instrument design. This motivates careful mapping design; it does not supply a quantified effect size for Core Resonance. For Core Resonance, the crucial mapping is not only hand-to-sound, but sound-to-world-to-sound. That extension is our design hypothesis, not a result of [1].

Savioja and Svensson [2] review geometrical acoustics for auralization and virtual environments, emphasizing reflection, diffraction and limitations of modeling approaches. Allen and Berkley [3] demonstrate an image-source simulation for rectangular rooms. Neither technique establishes an efficient solution for a changing galaxy-scale ecosystem with multiple moving sources; such generalization requires explicit approximations.

Frid and Ilsar [4] report a 118-person survey of digital music makers that highlights interface limitations, access requirements and enthusiasm for novel approaches. Their respondents are not Core Resonance users; this study supports asking accessibility and control questions, not projecting adoption.

3. Proposed two-layer prototype

Simulation state: an immutable resonance event containing emitter ID, time, 3D origin, frequency band, amplitude, travel speed, radius, lifetime and optional medium context. Receptive objects subscribe to spatial arrival tests; the event stream should remain independent of any glowing shell.

Presentation state: a shader or particle shell visualizes a pulse in three dimensions; audio mapping and a small receiver response make the interaction audible. A receiver located above, below or beyond the source plane must respond to the same logical event. Do not initially infer physical acoustic pressure or wave interference from this stylization.

Audit before construction: inspect the active Unity project, wave scripts, Nodes, JamBot triggers, MIDI/audio I/O and the currently working scene. Use a reversible branch or separate test scene. A declared engine version alone cannot verify a playable vertical slice. Limited inspection of JamBot indicates deterministic symbolic scheduling and motif recall; neural training has not been demonstrated by that inspection. This is a bounded implementation observation, not a comprehensive audit.

4. Study design and measurements

Use a randomized within-participant comparison of three instrument mappings: (A) audio-only reaction; (B) audiovisual 2D ripple; (C) audiovisual 3D pulse with a distinct above/below receiver. Keep musical triggers, volume, visuals and task length as comparable as practicable. Recruit novice and experienced music-makers; include alternative input accommodations.

For each mapping, ask participants to identify which source activated a receiver, predict the next receiver response, and intentionally create a target musical/environmental outcome. Metrics: receiver-prediction accuracy, task success, time-to-understanding, voluntary exploration, perceived agency, and input accessibility. Independently log audio output and event-arrival data. Pre-register primary outcomes before testing.

Run technical acceptance gates independently: positional correctness in all three axes; stable event payload and propagation timing; ability to process overlapping emitters; latency and frame-time profiling; no regression of existing JamBot/MIDI behavior. Report hardware, sample count, task order and failures.

5. Alternative explanations and limitations

A 3D effect may simply look more impressive; improved task ratings may reflect visual salience rather than better musical agency. Musicians may already understand spatial interaction more easily than novices. Sound quality, loudness or novelty may confound preference. Cross-media resonance metaphors should not be treated as physics laws.

A whole-world simulation with reflection, absorption, material memory, multi-agent fields and multiplayer remains future R&D. The proposed pulse system is an intentionally simplified game mechanic; whether it convincingly functions as an instrument is an empirical question.

6. Falsifiable predictions

H1: If spatial mapping improves legibility, receiver-prediction accuracy in C will exceed B after controlling for visibility of receiver targets. H2: Players will complete intentional world-to-sound tasks more often in B/C than A. H3 (possible null): adding 3D visuals may improve excitement without improving task accuracy or repeatable control.

A negative result warrants changing the mapping and feedback, not declaring the participant insufficiently musical. The smallest valid scientific contribution is a reproducible interface evaluation with independent task data.

Source provenance and research status

External literature establishes findings in related domains; it does not validate Nova Resonance implementations or outcomes. Project materials describe design intent. This paper reports no completed participant study or field pilot of the proposed system. The proposed protocols and hypotheses remain untested. No clinical, economic, cognitive or social effectiveness is established here.

Project provenance: this working paper draws on internal design briefs available to its editors. Those materials are not public evidence of runtime behavior or participant outcomes. Private operational details are omitted from this public working edition.

References

[1] Hunt, A., Wanderley, M. M., & Paradis, M. (2003). The importance of parameter mapping in electronic instrument design. Journal of New Music Research, 32(4), 429-440. https://doi.org/10.1076/jnmr.32.4.429.18853

[2] Savioja, L., & Svensson, U. P. (2015). Overview of geometrical room acoustic modeling techniques. Journal of the Acoustical Society of America, 138(2), 708-730. https://doi.org/10.1121/1.4926438

[3] Allen, J. B., & Berkley, D. A. (1979). Image method for efficiently simulating small-room acoustics. Journal of the Acoustical Society of America, 65(4), 943-950. https://doi.org/10.1121/1.382599

[4] Frid, E., & Ilsar, A. (2021). Reimagining (accessible) digital musical instruments: A survey on electronic music-making tools. NIME Proceedings. https://doi.org/10.21428/92fbeb44.c37a2370