[ad_1]
I’m engaged on including a characteristic to an present app, to take audio enter from the machine microphone, convert it to frequency area through a DFT, and ship it to a coreML mannequin.
The aim for the sign processing half is to acquire magnitude(dB) values throughout a selected variety of linear frequency bands (ex. 128) over a selected time frame outlined by the person. That is just like a spectrogram, however with out having to visualise it.
I am utilizing an ordinary AVAudioEngine to seize the microphone enter:
var audioEngine = AVAudioEngine()
let audioQueue = DispatchQueue(label: "com.queue.mic.ml", qos: .default, attributes: .concurrent, autoreleaseFrequency: .inherit)
When the person does a sure motion, microphone enter could be gathered, then transformed to a specified format and sampling charge, and at last transferred to the frequency area.
The present output format, as instance:
outputFormat = AVAudioFormat(commonFormat: .pcmFormatInt16,
sampleRate: Double(20000),
channels: 1, interleaved: true)!
The uncooked buffer channelDataValueArray is gathered by including a faucet to the microphone which appears to be working high-quality, following one other instance:
func startTappingMicrophone() {
// begin getting microphone enter and changing it to the format the coreML mannequin
// is educated for; put together for freq area
let inputNode = audioEngine.inputNode
let inputFormat = inputNode.outputFormat(forBus: 0)
guard let formatConverter = AVAudioConverter(from:inputFormat, to: self.outputFormat) else { return }
inputNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(bufferSize), format: inputFormat) { (buffer, time) in
self.audioQueue.async {
// An AVAudioConverter is used to transform the microphone enter to the format required for the mannequin.(pcm 16)
let pcmBuffer = AVAudioPCMBuffer(pcmFormat: self.outputFormat, frameCapacity: AVAudioFrameCount(self.outputFormat.sampleRate * 2.0))
var error: NSError? = nil
let inputBlock: AVAudioConverterInputBlock = {inNumPackets, outStatus in
outStatus.pointee = AVAudioConverterInputStatus.haveData
return buffer
}
// convert the format
formatConverter.convert(to: pcmBuffer!, error: &error, withInputFrom: inputBlock)
if error != nil {
print(error!.localizedDescription)
}
else if let channelData = pcmBuffer!.int16ChannelData {
let channelDataPointer = channelData.pointee
let channelDataValueArray = stride(from: 0,
to: Int(pcmBuffer!.frameLength),
by: buffer.stride).map{ channelDataPointer[$0] }
//Use channelDataValueArray for processing
self.processData(values: channelDataValueArray)
}
}
}
audioEngine.put together()
do {
strive audioEngine.begin()
}
catch {
print(error.localizedDescription)
}
}
Following Apple’s documentation, I attempted to transform every enter buffer by utilizing the Swift implementation of a discrete cosine remodel (DCT). Nonetheless, there appears to be a problem with this method, because the output values do not make sense.
Definition of DCT and buffers:
// outline numb. of frequency bands, ex: 128
forwardDCT = vDSP.DCT(rely: 128,
transformType: .II)!
/// A reusable array that accommodates the present body of time area audio information as single-precision values.
timeDomainBuffer = [Float](repeating: 0,
rely: bufferSize)
/// A reusable array that accommodates dB values of the freq. area values
filterBankBuffer = [Float](repeating: 0,
rely: filterBankCount)
Technique which makes an attempt to converts time area body information into frequency area information:
func processData(values: [Int16]) {
dispatchSemaphore.wait()
vDSP.convertElements(of: values,
to: &timeDomainBuffer)
vDSP.multiply(timeDomainBuffer,
hanningWindow,
consequence: &timeDomainBuffer)
forwardDCT.remodel(timeDomainBuffer,
consequence: &frequencyDomainBuffer)
/// be part of bands and convert to dB values
runFilterBank(freqDomainBuffer: frequencyDomainBuffer,
filtBankBuffer: &filterBankBuffer)
if frequencyDomainValues.rely >= 50 {
frequencyDomainValues.removeFirst(1) // preserve a max of bufferCount arrays
}
frequencyDomainValues.append(filterBankBuffer)
print(filterBankBuffer)
dispatchSemaphore.sign()
}
What’s the appropriate approach to convert the microphone information, to acquire magnitude(dB) values throughout a selected variety of linear frequency bands over a selected time frame(ex: 5 sec) outlined by the person?
[ad_2]
