Audio analysis and stem separation

Preview audio-to-MIDI transcription #

Open the file import menu and choose Audio to MIDI... to open Import Audio to MIDI. Enable Per-Note Audio first. The dialog transcribes the file on your computer and draws preview notes before you accept them. Files longer than 20 minutes are refused.

Pick a model from the Model list. Each installed MuScriptor model appears by size, for example MuScriptor small, mixed instruments, non-commercial. Transkun, piano is always listed. If no MuScriptor model is installed, the list shows MuScriptor (non-commercial): install a model from the Store and Transkun is selected. Picking a different model starts a new transcription.

The line under the Model list states the licence of the selected model. MuScriptor is for non-commercial use only (CC BY-NC 4.0). Transkun is MIT licensed and fine for commercial work.

MuScriptor models come from the Store under AI Models in small, medium and large sizes (about 0.4 GB, 1.2 GB and 5.5 GB). Small is the fastest. Medium is the default when installed. Large is the most accurate and slow without a GPU. Use MuScriptor for mixed material and Transkun for solo piano.

Before the first MuScriptor download, the Store shows the model licence from the MuScriptor authors, including that you must hold the rights to the audio you transcribe. Accept starts the download. Cancel downloads nothing, and the terms appear again next time. Accepting covers all three sizes, so you are asked once.

You can cancel a model download from the Store. A cancelled model stays uninstalled until a download finishes. The large model downloads in three parts. Parts that finished before you cancelled are kept, so the next download continues from the part that was interrupted.

Each engine needs a local Python runtime, downloaded once on first use. The dialog says so, and Accept becomes Install MuScriptor or Install Transkun. The MuScriptor runtime is about 1 GB. Setup needs Python 3.11 or 3.12 on the computer.

When transcription finishes, the dialog shows the note count and a short summary. MuScriptor lists the instruments it heard. It counts drum hits but does not import them. Transkun treats the source as piano and does not group instruments or produce tuning curves. Do not expect a full mix to turn into perfect notation.

Transkun also detects the sustain and soft pedals. Accept writes them to the track as CC64 (sustain) and CC67 (soft) automation lanes, aligned with the notes and snapped to the same Quantize grid, and shows those lanes. MuScriptor does not detect pedals.

Accept stays disabled until transcription finishes, then creates the notes, with the pedal lanes, as one Undo step. Cancel stops a running transcription or setup and closes the dialog without adding notes. Return and Escape are the accept and cancel shortcuts.

To convert an audio clip in the Arranger, right-click it and choose Convert Audio to MIDI. The same dialog opens as Convert Audio to MIDI, without the Quantize setting, and transcribes only the part of the file the clip plays. Accept replaces the audio clip with a MIDI clip whose notes play slices of the original audio. With Transkun, the pedals become CC64 and CC67 events inside the new clip. The conversion is one Undo step.

Draft: a Linux native walkthrough verified drop placement, preserved pitches, Undo, Redo and project reopening. A native macOS check on Apple Silicon installed MuScriptor small from the Store, set up its runtime, transcribed a test melody, accepted the notes and undid them. Windows remains unverified. For audio dropped into the Piano Roll, choose Transcribe to MIDI. Accept should place the detected phrase on the intended track from the dropped time position, preserving detected pitches, tuning and relative onset and duration spacing. The drop height should not transpose the phrase. Scrolling, zooming or changing selection during analysis should not move its captured track or time position.

Transcription controls #

The dialog has two settings: Model and Quantize. Neither model has sensitivity or threshold settings, so the result depends on the model you pick and the source audio.

Quantize starts at No Quantize. The other choices are 1/4 Note, 1/8 Note, 1/16 Note, and 1/32 Note. Changing it updates the preview without transcribing again. Inspect the unquantized timing first, then choose a grid if the phrase needs it.

Detect clip beats and warp audio #

Right-click an arranger audio clip and open Audio Warp. Detect Beats analyzes the source and adds confident beat markers. Warp to Project BPM then changes timing to follow the project. No Warp (1:1) restores an unwarped choice.

Audio Warp also offers Snap Markers to Grid, Clear All Beat Markers, and half or double beat-tempo interpretation. Beat Tempo x2 makes audio slower; Beat Tempo / 2 makes audio faster. Alt-click a marker to delete it. If the analysis marks half-time or double-time beats, correct that interpretation before accepting the warp.

Install Beat This... adds the optional local high-quality beat-analysis runtime. An installed runtime offers Repair Beat This... instead. Audio stays on this computer during the local analysis. Cancel Beat Analysis cancels a pending clip analysis. Beat detection and warping are separate actions; markers alone do not mean playback is tempo-synced.

Install and start Stem Separation beta #

Stem Separation is a Free beta add-on with an optional local runtime and model download. Its supported runtime package is currently Linux x86_64. Device choices in the dialog do not establish Windows or macOS support.

Enable Stem Separation in the Store and follow its runtime setup. Start with a short clip. In the arranger, right-click an existing audio clip and choose Separate Stems... . A new file drop can instead offer Extract Vocals..., Split Into 4 Stems..., or Stem Separation Settings... .

Separation analyzes a mixed recording to estimate sources. It can leave bleed, missing transients, or other artifacts. Keep the original clip and compare the resulting files before using them in a final mix.

Separation settings and results #

Choose Vocals + Instrumental or Vocals, Drums, Bass + Other. Engine offers Auto, SCNet XL IHF, and BS PolarFormer FP16. Device offers Auto, CPU, CUDA, MPS, and ONNX. Use Auto unless you know that the selected runtime and hardware support another choice.

Create grouped tracks and Mute original clip after import control the imported result. Create grouped tracks starts enabled. Mute original clip after import starts disabled. Separate starts the job. The processor keeps the job alive when you close the dialog; Close detaches the progress display and does not cancel separation.

Wait for Stems imported. The import checks output alignment and creates new clips at the source location. A failed job reports an error. Check the number of result tracks, their alignment, and the original clip mute state before playing the entire song.

Export stems renders existing project tracks. Stem Separation estimates sources from mixed audio. They are different workflows, even though both produce files called stems.