Skip to content

Show AudioEffectCapture by use of a shader showing the captured waveform in stereo #1172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

goatchurchprime
Copy link

Microphone data is normally read using AudioEffectCapture rather than AudioEffectRecord, so we should adapt this demo project to use it. And a good way to show that something is happening is to plot the waveform using a shader.

This shows it in stereo because a lot of computers have two microphones.

It's also useful to measure the sample rate of the microphone according to the data stream it is generating.

Screenshot from 2025-03-02 18-23-26

Comment on lines 9 to 14
var audiosamplesize : int = 882
var audiosampleframetextureimage : Image
var audiosampleframetexture : ImageTexture
var totalsamples = 0
var sampleduration = 0.0
var recordingbuffer = null
Copy link
Member

@Calinou Calinou Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use snake_case for all variable names, e.g. audio_sample_size.

Comment on lines 42 to 48
#recording = recordeffect.get_recording()
$PlayButton.disabled = false
$SaveButton.disabled = false
#recordeffect.set_recording_active(false)
#recording.set_mix_rate(mix_rate)
#recording.set_format(format)
#recording.set_stereo(stereo)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove commented out lines.

Copy link
Member

@Calinou Calinou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally on 4.4.stable (Linux), it works as expected. I've tested mic_record, not mic_input which fails with a script error on startup:

Parser Error: Static function "start_microphone()" not found in base "GDScriptNativeClass".

Is there a reason we have two demo projects now? They basically look identical at a surface, and we should try to keep the existing name for compatibility.

One issue I noticed in mic_record (but also mic_input) that the root node in the project (MicRecord) is rotated by 0.1 degrees, so the whole UI looks very slightly rotated. This is noticeable on the visualizer because it's not 100% straight.

@goatchurchprime
Copy link
Author

Many apologies for the extra mic_input project. It's WIP for testing out PR#105244.

This is a proposal for a new function Input.get_microphone_buffer(int p_frames) which is needed because the current method of:

AudioOutput <- AudioBus [ AudioCaptureEffect ] <- AudioStreamPlayer [ AudioStreamMicrophone ]

is unreliable because it locks two real-time serial buffers at two ends of the same chain, and any systematic drift between them eventually blows past any buffer.

@goatchurchprime
Copy link
Author

goatchurchprime commented Apr 16, 2025

I've rolled back the changes to the audio/mic_record demo in the hope that the AudioStreamMicrophone->AudioEffectRecord method of accessing the microphone will eventually become obsolete if PR#105244 gets accepted. The demo of the new interface with is in audio/mic_input.

I've also added in some shader code to visualize the sum of the left and right stereo microphones when progressively shifted in time. This is best tested with a pure tone of 400Hz, making a consistent 8 cycles across the 20ms sample window. To make this easy I've added in a drop-down option to generate these tones.

If you deploy this project to an Android phone so you can play the tone out loud while moving it to different positions about your stereo microphones, you get this kind of movie where you can see the waveforms shifting in amplitude and phase.

screencast.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants