Trying to load a sf2 file with asfxload returns "sfxload: no memory left"

Bug #183456 reported by Taupter
96
This bug affects 16 people
Affects Status Importance Assigned to Milestone
awesfx (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Binary package hint: awesfx

Trying to load a SoundFont2 file with asfxload returns "no memory left", even if the computer is plenty of RAM and the module is configured to majke 1GB available. Tested both with the package from the repository and a version compiled from unpatched sources, same behavior.

root@ptah:~# asfxload -V 100 "/storage/sdb1/Multimedia/SoundFonts/Soundfont 103.5Mg Real Font v2.1 Bank.sf2"
sfxload: no memory left

root@ptah:~# asfxload -M
DRAM memory left = 1037524 kB

root@ptah:~# lspci | grep Creative
00:0c.0 Multimedia audio controller: Creative Labs SB Audigy (rev 03)
00:0c.2 FireWire (IEEE 1394): Creative Labs SB Audigy FireWire Port

root@ptah:~# cat /etc/modprobe.d/alsa-base
alias snd-card-0 snd-emu10k1
alias snd-card-1 snd-via82xx
alias snd-card-2 snd-bt87x
alias snd-card-3 snd-usb-audio

# autoloader aliases
install sound-slot-0 /sbin/modprobe snd-card-0
install sound-slot-1 /sbin/modprobe snd-card-1
install sound-slot-2 /sbin/modprobe snd-card-2
install sound-slot-3 /sbin/modprobe snd-card-3
install sound-slot-4 /sbin/modprobe snd-card-4
install sound-slot-5 /sbin/modprobe snd-card-5
install sound-slot-6 /sbin/modprobe snd-card-6
install sound-slot-7 /sbin/modprobe snd-card-7

# Cause optional modules to be loaded above generic modules
install snd /sbin/modprobe --ignore-install snd && { /sbin/modprobe --quiet snd-ioctl32 ; : ; }
install snd-pcm /sbin/modprobe --ignore-install snd-pcm && { /sbin/modprobe --quiet snd-pcm-oss ; : ; }
install snd-mixer /sbin/modprobe --ignore-install snd-mixer && { /sbin/modprobe --quiet snd-mixer-oss ; : ; }
install snd-seq /sbin/modprobe --ignore-install snd-seq && { /sbin/modprobe --quiet snd-seq-midi ; /sbin/modprobe --quiet snd-seq-oss ; : ; }
install snd-rawmidi /sbin/modprobe --ignore-install snd-rawmidi && { /sbin/modprobe --quiet snd-seq-midi ; : ; }
# Cause optional modules to be loaded above sound card driver modules
install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 max_buffer_size=1024 enable_ir=1 $CMDLINE_OPTS && { /sbin/modprobe -Qb snd-emu10k1-synth ; }
install snd-via82xx /sbin/modprobe --ignore-install snd-via82xx $CMDLINE_OPTS && { /sbin/modprobe -Qb snd-seq ; }

# Load saa7134-alsa instead of saa7134 (which gets dragged in by it anyway)
install saa7134 /sbin/modprobe --ignore-install saa7134 $CMDLINE_OPTS && { /sbin/modprobe -Qb saa7134-alsa ; : ; }

# Load snd-seq for devices that don't have hardware midi;
# Ubuntu #26283, #43682, #56005; works around Ubuntu #34831 for
# non-Creative Labs PCI hardware
install snd /sbin/modprobe --ignore-install snd && { /sbin/modprobe -Qb snd-seq ; }
# Prevent abnormal drivers from grabbing index 0
options snd-emu10k1 index=0
options snd-via82xx index=1
options snd-bt87x index=2
options snd-usb-audio index=3
# options snd-bt87x index=-2
options cx88-alsa index=-2
options saa7134-alsa index=-2
options snd-atiixp-modem index=-2
options snd-intel8x0m index=-2
# options snd-via82xx-modem index=-2
# options snd-usb-audio index=-2
options snd-usb-usx2y index=-2
options snd-usb-caiaq index=-2
# Ubuntu #62691, enable MPU for snd-cmipci
options snd-cmipci mpu_port=0x330 fm_port=0x388

I'm running Kubuntu 7.10 on an AMD Athlon 64 3500+ (socket 939) processor.

root@ptah:~# uname -a ; free -m
Linux ptah 2.6.22-14-rt #1 SMP PREEMPT RT Tue Dec 18 06:37:06 UTC 2007 x86_64 GNU/Linux
             total used free shared buffers cached
Mem: 3906 3228 677 0 69 956
-/+ buffers/cache: 2202 1703
Swap: 1027 13 1013

root@ptah:~# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 47
model name : AMD Athlon(tm) 64 Processor 3500+
stepping : 2
cpu MHz : 2202.809
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow up pni lahf_lm
bogomips : 4409.40
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

Tags: patch
Revision history for this message
Taupter (taupter) wrote :

Just to add a bit of infrormation, I changed snd-emu10k1 module parameter to include max_buffer_size=1024 (supposedly allowing 1GB of ram for samples) _after_ I got the error. I increased it in steps (and rebooted after changing it to be sure the changes were applied) and tested again, same error.
I didn't test the -generic kernel as it is supposedly to work with -rt, as it's an internal wavetable hardware synthesizer and meant to be used to generate sound from MIDI events (that can be sent by an external MIDI master controller keyboard, as the CME-UF8 I own), routed by Rosegarden.
Qsynth/FluidSynth is a suboptimal workaround, as it's very processor-intensive and if you configure it to have a low latency you'll get a lot of "pops" and other sound artifacts, hardly performance-quality-wise. Upgrading machine atm is out of question. ;)

Revision history for this message
Taupter (taupter) wrote :

I found something funny about it:

With 4GB DDR RAM (four 1GB banks):
asfxload succeeds loading an 8MB Piano SoundFont
asfxload fails with no memory left when loading a 103MB General MIDI SoundFornt

With 2GB DDR RAM (two 1GB banks):
asfxload succeeds loading an 8MB Piano SoundFont
asfxload succeeds loading a 103MB General MIDI SoundFornt

Both tests were done after a cold boot, switching to a console and logging as root.

Revision history for this message
Daniel T Chen (crimsun) wrote :

Is this symptom still reproducible in 8.10 or 9.04?

Changed in awesfx:
status: New → Incomplete
Revision history for this message
Al Buntu (biedermann2) wrote : Re: [Bug 183456] Re: Trying to load a sf2 file with asfxload returns "sfxload: no memory left"

Yes - it is reproducable in 8.10.

Am Mittwoch, den 26.11.2008, 05:59 +0000 schrieb Daniel T Chen:

> Is this symptom still reproducible in 8.10 or 9.04?
>
> ** Changed in: awesfx (Ubuntu)
> Status: New => Incomplete
>

Revision history for this message
Marcus Brinkmann (marcus-brinkmann) wrote :

Same here with Ubuntu 8.10 amd64 and 4GB Ram. Worked fine with Ubuntu 8.04 32bit on the same hardware (assumingly because then only 3.5 GB RAM are available).

I debugged asfxload and traced it down to an error code from the SNDRV_EMUX_IOCTL_LOAD_PATCH ioctl in seq_load_patch. Curiously, smallish soundfonts work. There are no kernel messages related to this, and I don't feel like debugging the kernel right now. :)

Revision history for this message
Robert Mahoney (mahoneyr) wrote :

Same here, identical setup to Marcus.

Revision history for this message
Taupter (taupter) wrote :

Someone sent me an e-mail explaining that maybe the problem can be related to iommu and a failure in the dma_alloc_coherent() call, and told me to include "iommu=force" in the kernel's boot parameters. It didn't work with my computer, but nevertheless it may work on your specific setup, so please try and report here.
At least this can bring some light to Ubuntu developers.

Revision history for this message
Robert Mahoney (mahoneyr) wrote :

Is "incomplete" still appropriate for this bug? It is still present in current distributions, does not appear to be a duplicate, and should not be allowed to expire.

Changed in awesfx:
status: Incomplete → New
Revision history for this message
Michael Allen (mike43202) wrote :

Can't anyone pick this up? Please? I have had this problem for years now, ever since I upped my RAM. I would love to use my emu10k1 synth chip again since it is less laggy than any software synth.

I was told that this problem is due to the EMU10k1 being unable to address more than 2G of memory. That is why the emu10k1 driver can load a soundfont when the system has 2G, but not when it has more. I had 4G and now 6G and so I have this issue.

It appears that the solution would be to modify the emu10k1 driver so it allocates its soundfont cache within the lowest 2G of RAM. I believe the allocation is done in emu10k1_main.c, and it is currently done by vmalloc. I have read of references to a DMA32 flag that is often used to fix similar problems with other old sound cards.

Comments?

Revision history for this message
Thomas Richter (thor-math) wrote :

The problem with this goes a bit deeper. The reason for all this trouble is that the emu10k1 chipset requires DMA from main memory to pick up the samples, and - unfortunately - the DMA engine of the chip is broken. Instead of using the full range of 32 bit DMA available on the PCI bus, it only addresses 31 bit of the 32 available.

To work around this problem, the kernel driver first allocates a page of main memory, signaling the 32 bit DMA memory "zone", i.e. indicating the kernel that it requires 32 bit memory in a 64 bit memory system. However, if now the returned memory happens to have bit 31 set, the emu10k1 wouldn't be able to address the memory. Then, the driver releases the memory again, and requests memory in the 16MB "ISA" memory region, which is clearly suitable for the chip. Unfortunately, this type of memory runs out easily, and then the chip is left with no usable memory at all.

The problem is that the kernel isn't smart enough to provide memory that is "just right" for the chip, i.e. the "DMA zone" allocation isn't quite sufficient for this situation. There seem to be plans on the LKLM to address this, but it's unclear how long this will take - after all, the complete kernel page allocator must be re-structured to fix this.

How to work around? Oh well, the easiest is to lower the 32bit PCI DMA zone from 4G to 2G. This means that some non-broken devices will require to allocate pages in the 2GB region (bad enough), and the memory layout might be sub-optimal for those devices, but at least the hack will work.

For that, download the kernel, and edit arch/x86/include/asm/dma.h, locate the line:

#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)

and replace the 4UL by a 2UL, which will shrink the 32-bit DMA zone by half. This is not a permanent solution - it's an ugly bad hack that is "good enough for me". There is no kernel boot option to do this AFAIK.

So long, happy hacking,

Thomas

Revision history for this message
Michael Allen (mike43202) wrote : RE: [Bug 183456] Re: Trying to load a sf2 file with asfxload returns "sfxload: no memory left"
Download full text (6.7 KiB)

Thanks, this helps a lot.

But what if the driver just makes repeated requests until it gets the memory in the range it needs, by chance? Not likely?

-----Original Message-----
From: Thomas Richter <email address hidden>
Sent: Sunday, March 29, 2009 6:38 PM
To: <email address hidden>
Subject: [Bug 183456] Re: Trying to load a sf2 file with asfxload returns "sfxload: no memory left"

The problem with this goes a bit deeper. The reason for all this trouble
is that the emu10k1 chipset requires DMA from main memory to pick up the
samples, and - unfortunately - the DMA engine of the chip is broken.
Instead of using the full range of 32 bit DMA available on the PCI bus,
it only addresses 31 bit of the 32 available.

To work around this problem, the kernel driver first allocates a page of
main memory, signaling the 32 bit DMA memory "zone", i.e. indicating the
kernel that it requires 32 bit memory in a 64 bit memory system.
However, if now the returned memory happens to have bit 31 set, the
emu10k1 wouldn't be able to address the memory. Then, the driver
releases the memory again, and requests memory in the 16MB "ISA" memory
region, which is clearly suitable for the chip. Unfortunately, this type
of memory runs out easily, and then the chip is left with no usable
memory at all.

The problem is that the kernel isn't smart enough to provide memory that
is "just right" for the chip, i.e. the "DMA zone" allocation isn't quite
sufficient for this situation. There seem to be plans on the LKLM to
address this, but it's unclear how long this will take - after all, the
complete kernel page allocator must be re-structured to fix this.

How to work around? Oh well, the easiest is to lower the 32bit PCI DMA
zone from 4G to 2G. This means that some non-broken devices will require
to allocate pages in the 2GB region (bad enough), and the memory layout
might be sub-optimal for those devices, but at least the hack will work.

For that, download the kernel, and edit arch/x86/include/asm/dma.h,
locate the line:

#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)

and replace the 4UL by a 2UL, which will shrink the 32-bit DMA zone by
half. This is not a permanent solution - it's an ugly bad hack that is
"good enough for me". There is no kernel boot option to do this AFAIK.

So long, happy hacking,

Thomas

--
Trying to load a sf2 file with asfxload returns "sfxload: no memory left"
https://bugs.launchpad.net/bugs/183456
You received this bug notification because you are a direct subscriber
of the bug.

Status in “awesfx” source package in Ubuntu: New

Bug description:
Binary package hint: awesfx

Trying to load a SoundFont2 file with asfxload returns "no memory left", even if the computer is plenty of RAM and the module is configured to majke 1GB available. Tested both with the package from the repository and a version compiled from unpatched sources, same behavior.

root@ptah:~# asfxload -V 100 "/storage/sdb1/Multimedia/SoundFonts/Soundfont 103.5Mg Real Font v2.1 Bank.sf2"
sfxload: no memory left

root@ptah:~# asfxload -M
DRAM memory left = 1037524 kB

root@ptah:~# lspci | grep Creative
00:0c.0 Multimedia audio controller: Creative Labs SB Audi...

Read more...

Revision history for this message
Thomas Richter (thor-math) wrote :

Well, simply re-allocating doesn't help since this will get it exactly the same memory as before - so it had to fetch memory over and over again until it succeeds, and then had to release all the unusable memory again. Possible, but would require it to fetch at most 2GB of memory (urgh!) just for nothing, plus the overhead, and the time needed to do the allocation and to loop for all that to happen. Remember, these are 2G/4K = 524288 allocations to be done!

I'd say the problem should really fixed at its origin. For first, the emu10k1 is broken - too bad, a rather good card otherwise - I've two of them. And if replacing that is not an option (which it isn't for me), I'd say that the kernel should really provide better mechanisms to allocate the right memory.

So long,
Thomas

Revision history for this message
kai (kappa8086) wrote :

Seems a different cause in amd64 platform.
In that a small sf2 file can be loaded successfully, (very few megabytes) it is said about the IOMMU

Revision history for this message
Thomas Richter (thor-math) wrote :

I don't quite understand your comment. The above "fix" applies to the AMD64 platform. There is no such thing as a 32bit DMA zone on x86, but still the kernel might want to allocate physical addresses for the buffer memory beyond the 2GB barrier (bit 31 set), thus it is prone to failure in any system equipped with more than 2GB of RAM. Sure enough, a *small* soundfont could be loaded if enough memory is available in the 16MB DMA zone, but this depends on a lot of other factors (as, for example, which other devices require DMA memory).

So long,
Thomas

Revision history for this message
kai (kappa8086) wrote :

Yes, indeed. As in windows x86 the driver leads a bluescreen but in x64 it doesn't, I thought there's a solution in x64 and it could do the same as windows, then tried yesterday to use iommu=force (soft no effect) parameter, however, instead of "no memory left", asfxload crashed with some unrecognizable kernel codes. sigh

Revision history for this message
simotsa (simotsa) wrote :

Using gentoo-sources-2.6.36-r3.

Changing arch/x86/include/asm/dma.h fixed it for me, but I don't know if this change would affect system stability.

Revision history for this message
Thomas Richter (thor-math) wrote : Re: [Bug 183456] Re: Trying to load a sf2 file with asfxload returns "sfxload: no memory left"

SnEptUne wrote:
> Using gentoo-sources-2.6.36-r3.
>
> Changing arch/x86/include/asm/dma.h fixed it for me, but I don't know if
> this change would affect system stability.

Not for me, at least. I'm working with this configuration for more than
a year now without any bad side effects. What will happen, of course, is
that the kernel will copy data outside of the (then smaller) 2GB DMA
region to the 2GB DMA region, even for hardware that is fully 32-bit DMA
capable. Note that hardware that supports 64 bit addresses for DMA is
unaffected.

Greetings,
 Thomas

Revision history for this message
Raymond (superquad-vortex2) wrote :

you have to change max_buffer_size if you want to load a larger soundfont as default value is 128Mb

http://git.alsa-project.org/?p=alsa-kernel.git;a=blob_plain;f=Documentation/sound/alsa/ALSA-Configuration.txt

  Module snd-emu10k1
  ------------------

    Module for EMU10K1/EMU10k2 based PCI sound cards.
   * Sound Blaster Live!
   * Sound Blaster PCI 512
   * Emu APS (partially supported)
   * Sound Blaster Audigy

    extin - bitmap of available external inputs for FX8010 (see bellow)
    extout - bitmap of available external outputs for FX8010 (see bellow)
    seq_ports - allocated sequencer ports (4 by default)
    max_synth_voices - limit of voices used for wavetable (64 by default)
    max_buffer_size - specifies the maximum size of wavetable/pcm buffers
                       given in MB unit. Default value is 128.
    enable_ir - enable IR

Revision history for this message
Helmut Duregger (helmutduregger) wrote :

As mentioned in the original ticket and the first comment (https://bugs.launchpad.net/ubuntu/+source/awesfx/+bug/183456/comments/1) changing max_buffer_size does not help.

The soundfonts that fail to load are not even close to the 128MB default (at least in file size), e.g.

~30MB file fails to load, seems to load partially since some DRAM memory is lost in the process.

$ asfxload -M
DRAM memory left = 131068 kB
$ asfxload ./soundfont.SF2
sfxload: no memory left
$ asfxload -M
DRAM memory left = 114902 kB
$ ls -l ./soundfont.SF2
-rwxrwxrwx 1 user group 29258148 2010-01-23 16:49 ./soundfont.SF2

This is still relevant in Ubuntu 10.10.

Revision history for this message
Helmut Duregger (helmutduregger) wrote :

Using 64bit and 4GB of RAM, Audigy 4 (non-pro).

Revision history for this message
Thomas Richter (thor-math) wrote :

Raymond wrote:
> you have to change max_buffer_size if you want to load a larger
> soundfont as default value is 128Mb
The issue is not with the size of the soundfont. The issue is that the
kernel tries to put them into the 16MB ISA
DMA zone, being short of memory below 2GB.

Here's the technical explanation:

The kernel first tries to allocate 32bit memory for the soundfont. It
then checks whether the allocated memory is
above the 2GB barrier. Having a hardware bug in the emu10k1, the
hardware can only support DMA from the lowest 2GB
of memory because it simply ignores bit 31 of the address. The memory
allocator then comes back with memory which is,
due to the memory allocation policy, *above* the 2GB barrier whenever
more than 2GB is installed in the system.
This, of course, won't work, and the kernel/emu10k1 driver notices this,
releases the memory and then tries next to take
the memory from the 16MB ISA zone. Of course, there isn't enough memory
available to load the soundfont, which then
fails.

There is a fix, namely shorten the kernel 32bit DMA zone to 2GB, namely
replace in arch/x86/include/asm.h the following
line:

#define MAX_DMA32_PFN ((2UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)

here the 4UL has been replaced by a 2UL, which does the indicated
reduction of the 32bit DMA memory, and brings the
emu10k1 back to work. It has no other drawbacks, i.e. the machine is
working with correspondingly patched kernels
for over a year by now.

The true fix would be to make the memory allocation of the kernel a bit
more flexible than it is now.

Greetings,
    Thomas

Revision history for this message
Raymond (superquad-vortex2) wrote :

I can load more than 16Mb soundfont into my sb live! platinum ct4760p on my 32bits machines, so Is this bug specific to 64bits machine only

asfxload -M
DRAM memory left = 131068 kB
asfxload -M PCLite.sf2
DRAM memory left = 100523 kB

Revision history for this message
Thomas Richter (thor-math) wrote :

Raymond wrote:
> I can load more than 16Mb soundfont into my sb live! platinum ct4760p on
> my 32bits machines, so Is this bug specific to 64bits machine only
>

No, it is specific to machines with more than 2GB memory. Please see my
report.

Merry Christmas

Thomas

Revision history for this message
Daniel Richard G. (skunk) wrote :

Thomas, thank you for your clear description of the issue.

Given that the problem is memory beyond the 2GB barrier, something I wanted to try was mmap()'s MAP_32BIT flag. You can sort-of do a malloc() by making an anonymous mapping, and so I hacked up safe_malloc() and safe_free() (in awelib/malloc.c) accordingly. My patch against the awesfx-0.5.1a source (current Ubuntu version) is attached.

The addresses returned are consistently 0x4[0-9a-f]{7}, yet I continue seeing the "no memory left" error. Do you think there is something in this approach that may allow a solution that doesn't require kernel modification, or are we still at the mercy of the kernel's memory allocation policy?

Revision history for this message
Thomas Richter (thor-math) wrote :

Daniel Richard G. wrote:
> Thomas, thank you for your clear description of the issue.
>
> Given that the problem is memory beyond the 2GB barrier, something I
> wanted to try was mmap()'s MAP_32BIT flag. You can sort-of do a malloc()
> by making an anonymous mapping, and so I hacked up safe_malloc() and
> safe_free() (in awelib/malloc.c) accordingly. My patch against the
> awesfx-0.5.1a source (current Ubuntu version) is attached.
>
> The addresses returned are consistently 0x4[0-9a-f]{7}, yet I continue
> seeing the "no memory left" error. Do you think there is something in
> this approach that may allow a solution that doesn't require kernel
> modification, or are we still at the mercy of the kernel's memory
> allocation policy?
Not really, at least not according to my understanding of the situation.
Look,
obviously the kernel has to copy over the sound font from user memory to
kernel
memory anyhow - simply because the user memory where awelib initially
loads the sound font into is released by the Os as soon as the program
is unloaded.
Thus, any user space tools cannot, by construction, control where the
wavetable
will go. It is the kernel that runs out of memory when trying to
allocate persistent
kernel memory for the sound font, not the user space tools.

Thus, this really requires a modification of the kernel emu10k1 driver,
unfortunately.

Greetings,
    Thomas

tags: added: patch
Revision history for this message
Daniel Richard G. (skunk) wrote :

Hm, I was under the impression that the driver/emu10k1 simply read the sound font data from user memory, and saved it to dedicated memory belonging to the emu10k1.

Ah well, at least that answers that. Thanks for writing.

Changed in awesfx (Ubuntu):
status: New → Confirmed
Revision history for this message
quequotion (quequotion) wrote :

This bug must be affecting far more than 9 people.

Revision history for this message
Thomas Richter (thor-math) wrote :

It most certainly does. However:

Current kernel hackers seem to prefer to work around the situation by reverting to a timidity based software emulation of midi. It is not exactly the same as the emu10k1 midi, and - if you ask me - is of lower quality. Thus, at least kernel developers have a reason to say "go away, we don't need to care".

Second, fixing the issue requires a deeper modification of the kernel, namely providing a more flexible memory allocation, or more DMA zones. The above hack does the trick, and it doesn't affect stability, but it changes the DMA zones of course - and just doing that for a single card might not be a good argument. Instead, the kernel should provide means to allocate memory with more flexible constraints than just the DMA zones that exist now.

However, the issue doesn't seem to be pressing enough to move forward - at least this is my impression.

Any kernel developers reading this here and could comment on this?

Revision history for this message
Daniel Richard G. (skunk) wrote :

Thomas,

In the dma.h header you've cited, I see that the MAX_DMA32_PFN definition is commented thusly:

    /* 4GB broken PCI/AGP hardware bus master zone */
    #define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)

Given that this value is intended for broken (PCI) hardware, and the emu10k1 does fall into that category, could the argument be made that halving it would make it apply more broadly? There could well be other hardware that fails in the 4GB range, but works in 2GB.

Revision history for this message
Thomas Richter (thor-math) wrote :

I would say so, yes. I just don't have any other concrete examples where this might be the case.

Revision history for this message
Raymond (superquad-vortex2) wrote :

refer to alsa-kernel/include/emu10k1.h

/* FIXME? - according to the OSS driver the EMU10K1 needs a 29 bit DMA mask */
#define EMU10K1_DMA_MASK 0x7fffffffUL /* 31bit */
#define AUDIGY_DMA_MASK 0x7fffffffUL /* 31bit FIXME - 32 should work? */

Revision history for this message
simotsa (simotsa) wrote :

Anyone know which file to patch for kernel 3.0.x?

Revision history for this message
koppi (jakob-flierl) wrote :

Wrote a small tutorial "HOWTO setup EMU10k soundfont DSP on Ubuntu 11.10" see here: https://github.com/koppi/renoise-refcards/wiki/Howto-setup-EMU10k-soundfont-DSP-on-Ubuntu-11.10 Feel free to edit the Wiki page at GitHub or send me your commends.

Revision history for this message
quequotion (quequotion) wrote :

>>Raymond

I didn't understand what your post was about at first.

After looking into it, this does seem like the single line of code to change.

From OSS (via a page so outdated it exists only in google cache):

linux/sound/oss/emu10k1/main.c
...
121 /* the emu10k1 _seems_ to only supports 29 bit (512MiB) bit bus master */
122 #define EMU10K1_DMA_MASK 0x1fffffff /* DMA buffer mask for pci_alloc_consist */
...

Perhaps the OSS developers were right.

To do:

1. Patch a current kernel to give a 29bit DMA mask to the emu10k1 module
2. Test emu10k1 card and awesfx against the patched kernel.
3. If successful, push the patch upstream and into the kernel.

This change sounds a lot better to me as it should not affect any other hardware.

A small (basically one line) patch with no side effects should not have much trouble getting into the kernel.

Revision history for this message
Thomas Richter (thor-math) wrote :

The problem is not which DMA mask the kernel assigns to the module. The problem is what the kernel does if the memory to be accessed via the emu10k1 DMA is *not* in range of the DMA mask. The emu10k1 reaches the 30 bit address space just fine, as my all-day experience tells me.

What the kernel driver does is in principle correct - it goes for the 24 bit DMA space (the lowest 16 MB or so). This address range would be reachable by the emu10k1. The problem is just that this memory region is too small to hold a sound font, and then simply fails. The granularity offered by the kernel is too small - there is either the 64 bit space (which is not reachable by PCI anyhow), then the 4GB space, still too large for the emu10k1, and then the 16MB ISA space, which is reachable, but too small to be useful. Thus, the kernel memory architecture is too unflexible.

Revision history for this message
Daniel Richard G. (skunk) wrote :

Hi Thomas,

You've previously said that the kernel hackers aren't overly willing to put in a proper fix for this issue. I don't see an entry for this bug in the Linux kernel bugzilla, however (https://bugzilla.kernel.org/), so it's possible they aren't even aware of it.

Regardless of whether or not it is likely to be followed up, it would be good to have an upstream bug report linked to this one, and you've clearly demonstrated your grasp of what the problem is and what the fix needs to be. Would you have a few moments to file a bug with the upstream, and link it here?

Revision history for this message
Thomas Richter (thor-math) wrote :

Sorry, but I don't waste my time there. I believe I reported this problem already several years ago, and it was basically ignored or the patch wasn't accepted. To some degree, I understand this because it doesn't address the *real* problem (which is the unflexibility of the kernel memory allocation mechanism), but then again, I'm not knowledgable enough to provide a fix for that (namely a re-design of the kernel allocation). So no-go, and it went nowhere.

If you want to make your own experiences with the helpfulness of the linux kernel team, you surely have my appreciation, please go ahead and report it. For all others that just want to have a working sound blaster and don't care about arguing with the kernel hackers, a patch is here, even instructions how to apply it thanks to Jakob.

Revision history for this message
Raymond (superquad-vortex2) wrote :
Revision history for this message
quequotion (quequotion) wrote :

ohhh...

this is a bit deeper than my knowledge of the linux kernel but i'm starting to get the picture.

perhaps there is another way to get this done, without proposing major changes to the kernel.

is it possible to add code to either the emu10k1 module or asfxload to buffer large soundfonts?

The goal being to load large soundfonts without breaking memory allocation by limiting how much the emu10k1 can load into memory and caching the rest to another memory location (faster than to disk).

if soundfont data could be loaded and unloaded on-the-fly, as needed, it may be possible to use large soundfonts without such a large DMA.

Revision history for this message
Joe M (someone13) wrote :

An alternative solution to patching the kernel is to add a line to the bootloader excluding all but 2GB of RAM whenever you're going to be using MIDI. I got this idea from reading the forum at http://ubuntuforums.org/showthread.php?t=1340309
 and have confirmed that it works. I have 8GB of RAM and a SoundBlaster Audigy 2 ZS card.

I'm using the version of GRUB that comes with Ubuntu 12.04. I pressed "e" at the boot loader screen to add custom options and, after the word "splash", added the line "memmap=2048M\\$6144M " with no quotes. I've confirmed that I'm able to load a SoundFont of in excess of 100MB with no problem whereas previously I got the error message. MIDIs play fine with the large SoundFont in Rosegarden.

This is a kludge, but at least it's not modifying a system-wide kernel variable. Note that when you boot up with that option, your system will of course think that it only has 2GB of RAM.

Many thanks to sdowney717 for coming up with this solution in a different context.

Revision history for this message
quequotion (quequotion) wrote :

Another approach...

I set out to target the 2GB limit workaround to only affect the emu10k1, and at the same time make some placeholders for code to create an additional 31bit DMA region. Then I spent hours trying to set the right kernel configuration for my system.... and gave up.

This patch is against Linus's git branch and probably doesn't work, but is useful to bookmark some of the places where changes have to be made. I'm going to do a more simple test tomorrow with the source of my current kernel (3.5.0-15.22) to see if this can be worked out. In the mean time, if anyone has time to kill I'd love to know if this even compiles.

Revision history for this message
quequotion (quequotion) wrote :

My patch will not compile and fails with:

include/linux/mmzone.h:335:2: error: #error ZONES_SHIFT -- too many zones configured adjust calculation

Looking into this, I found in include/linux/gfp.h:

 * GFP_ZONE_TABLE is a word size bitstring that is used for looking up the
 * zone to use given the lowest 4 bits of gfp_t. Entries are ZONE_SHIFT long
 * and there are 16 of them to cover all possible combinations of
 * __GFP_DMA, __GFP_DMA32, __GFP_MOVABLE and __GFP_HIGHMEM.

So to add another zone I'd need to increase that to 25 (I think) and this is way over my head.

I can see why no one is jumping for the chance to fix this bug, but I hope someone out there might have an idea what to do. My approach is pretty naieve and ultimately not the best idea. Ideally, there would be a subzone of the DMA32 zone applied to specific PCI devices like the emu10k1. If this could be done without redefining the entire DMA system it would be wonderful.

Revision history for this message
Harald Judt (hjudt) wrote :

I have tried the small "2L" change by Thomas Richter and it works fine, no negative side effects so far. Apparently, the 4GB barrier was a bit of an arbitrary value anyway, you can read about that here: https://lwn.net/Articles/152337/

I found the memmap=2048M\\$6144M "workaround" proposed on http://www.alsa-project.org/main/index.php/Matrix:Module-emu10k1 rather ridiculous, even if it doesn't require patching.

With Thomas' simple fix making it rather easy for us to solve our problem, let's hope the kernel devs won't change the MAX_DMA32_PFN code too soon so that it doesn't apply anymore. Because I do not believe they will care about 1 or 2 pieces of hardware that will eventually fade into history on all new non-PCI systems... Still, isn't it quite an achievement that we can still use a soundcard that has been released ages ago?

BTW: The changes no longer have to be done in arch/x86/include/asm.h but in arch/x86/include/asm/dma.h for recent kernels.

Revision history for this message
pzad (pzubaj) wrote :

Looks like audigy (and probably emu10k1 too) supports 32 bits DMA. There are two addressing modes, where first supports 32 MB space in 2 GB memory and 8192 pages (this is default mode). Second is 16 MB space in 4 GB and 4096 pages. I tested second one an looks like it is working (audigy 8GB ram, 64 bit).

Revision history for this message
Thomas Richter (thor-math) wrote :

If you have an idea how to activate the 16MB mode for the emu10k1, then this would be pretty helpful.

Revision history for this message
imachine (m-jedrasik) wrote :

It's 2018 and this bug is still here; running 4GB ram 64bit + Live 5.1 and the effect is just like here.

I have an onboard builtin "soundcard" but the sound it produces is way flatter and less full than the one I hear via the SB Live; may be just a case of config; maybe there is a difference in the hardware and that's all that can be done about it.

I rather utilize the SB Live anyway; loading soundfonts up to 16MB works fine (fonts like Aspirin I find to be most interesting so far); I do want to load at least Fluid fonts though to at least mimic the basic sound I can obtain via software.

Maybe this oughta be a bug against the kernel, or some alsa package, not the awesfx package ?

Revision history for this message
Wolfe Gothmog (wolfegothmog) wrote :

This bug still exists and is easily reproducible in the latest version of Ubuntu. I have a PC with 32GB of RAM and I can't load a Soundfont bigger than 16mb with asfxload, on my PC with 2GB of RAM I can load a 600mb Soundfont without issue, this really needs to be fixed as using a sofsynth causes pops and glitches with my setup unfortunately.

Revision history for this message
Taupter (taupter) wrote :

More than ten years and this bug is still around and biting me.
Those SoundBlaster cards are really made to last.

Revision history for this message
Thomas Richter (thor-math) wrote :

Look, what else to say? The patch is there, it has been confirmed to work, and it still works here. See #10 on this list.

If the Ubuntu team does not want to pick it up, nor the kernel team wants to pick it up, nothing will happen. If anyone is having a better fix, also go along. Until then, see #10.

Revision history for this message
Wolfe Gothmog (wolfegothmog) wrote :

I tried the patch and it works great (though I can't seem to get the other patch to work to increase the limit to 2gb for soundfonts), it's just really inconvenient to recompile the kernel and now after initially compiling it, it doesn't want to compile again and complains about a signing key. Anyways patch does work great thanks everyone (and also does anyone know how to get a linux-image-extra and linux-headers-all when compiling the kernel I only get linux-image and linux-headers)

Revision history for this message
Raffaello (milanraf) wrote :

Bug still there

Ubuntu Studio 20.04
Linux studio 5.4.0-170-lowlatency #188-Ubuntu SMP PREEMPT

asfxload ver.0.5.2

Reproduced on both CT4620 and CT4832

Recompiling kernel is not an option for me

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.