我可以将librosa生成的频谱图转换回音频吗? [英] Can I convert spectrograms generated with librosa back to audio?

问题描述

我将一些音频文件转换为频谱图，并使用以下代码将其保存到文件中:

I converted some audio files to spectrograms and saved them to files using the following code:

import os
from matplotlib import pyplot as plt
import librosa
import librosa.display
import IPython.display as ipd

audio_fpath = "./audios/"
spectrograms_path = "./spectrograms/"
audio_clips = os.listdir(audio_fpath)

def generate_spectrogram(x, sr, save_name):
    X = librosa.stft(x)
    Xdb = librosa.amplitude_to_db(abs(X))
    fig = plt.figure(figsize=(20, 20), dpi=1000, frameon=False)
    ax = fig.add_axes([0, 0, 1, 1], frameon=False)
    ax.axis('off')
    librosa.display.specshow(Xdb, sr=sr, cmap='gray', x_axis='time', y_axis='hz')
    plt.savefig(save_name, quality=100, bbox_inches=0, pad_inches=0)
    librosa.cache.clear()

for i in audio_clips:
    audio_fpath = "./audios/"
    spectrograms_path = "./spectrograms/"
    audio_length = librosa.get_duration(filename=audio_fpath + i)
    j=60
    while j < audio_length:
        x, sr = librosa.load(audio_fpath + i, offset=j-60, duration=60)
        save_name = spectrograms_path + i + str(j) + ".jpg"
        generate_spectrogram(x, sr, save_name)
        j += 60
        if j >= audio_length:
            j = audio_length
            x, sr = librosa.load(audio_fpath + i, offset=j-60, duration=60)
            save_name = spectrograms_path + i + str(j) + ".jpg"
            generate_spectrogram(x, sr, save_name)

我希望保留音频的最详细信息和质量，以便我可以将它们重新转换为音频而不会造成太多损失(它们每个为80MB).

I wanted to keep the most detail and quality from the audios, so that i could turn them back to audio without too much loss (They are 80MB each).

是否可以将它们重新转换为音频文件?我该怎么办?

Is it possible to turn them back to audio files? How can I do it?

我尝试使用librosa.feature.inverse.mel_to_audio，但是它不起作用，并且我认为它不适用.

I tried using librosa.feature.inverse.mel_to_audio, but it didn't work, and I don't think it applies.

我现在有1300个频谱图文件，并希望与它们一起训练生成对抗网络，以便可以生成新的音频，但是如果以后无法收听结果，我就不想这样做./p>

I now have 1300 spectrogram files and want to train a Generative Adversarial Network with them, so that I can generate new audios, but I don't want to do it if i wont be able to listen to the results later.

推荐答案

是的，可以恢复大部分信号并使用例如Griffin-Lim算法(GLA).可以在 librosa 中找到其针对Python的快速"实现.使用方法如下:

Yes, it is possible to recover most of the signal and estimate the phase with e.g. Griffin-Lim Algorithm (GLA). Its "fast" implementation for Python can be found in librosa. Here's how you can use it:

import numpy as np
import librosa

y, sr = librosa.load(librosa.util.example_audio_file(), duration=10)
S = np.abs(librosa.stft(y))
y_inv = librosa.griffinlim(S)

这就是原始和重建的样子:

And that's how the original and reconstruction look like:

默认情况下，该算法会随机初始化相位，然后反复进行正向和反向STFT操作以估计相位.

The algorithm by default randomly initialises the phases and then iterates forward and inverse STFT operations to estimate the phases.

查看您的代码以重构信号，您只需要执行以下操作:

Looking at your code, to reconstruct the signal, you'd just need to do:

import numpy as np

X_inv = librosa.griffinlim(np.abs(X))

这只是一个例子.正如@PaulR所指出的，在您的情况下，您需要从jpeg加载数据(这是有损的！)，然后首先对amplitude_to_db应用逆变换.

It's just an example of course. As pointed out by @PaulR, in your case you'd need to load the data from jpeg (which is lossy!) and then apply inverse transform to amplitude_to_db first.

由于人工神经网络的进步，可以进一步改善算法，尤其是相位估计. 此处是讨论某些增强功能的论文.

The algorithm, especially the phase estimation, can be further improved thanks to advances in artificial neural networks. Here is one paper that discusses some enhancements.

这篇关于我可以将librosa生成的频谱图转换回音频吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！