0.9u-12 --> Hiccups and Clicks

[edward]

Peter - I'm still getting random hiccups and clicks between tracks (as discussed already) - not exactly smooth gapless. 

As before, it happens on long playlists, but if I stop and play the "problem" songs individualy then all is fine.

I know you've been pulling your hair out over this one, and have been alluding to the possibility that it has to do with my setup - but it only all started with 0.9u-7 when you introduced the 'Mem' checkbox. And FWIW, I've played with the possibility of different Scheme and Invert (and subsequently changed my preferences - see sig) but with no resolution.

Do you have any other ideas? 

I seem to remember being able to play long playlists yesterday with 0.9u-11 (when I was experimenting with 'DAC is 16/44.1' and no Double). Could that be the culprit? It appears to happen during the "preprocessing" when it loads the next track.

[PeterSt]

I think I started myself talking about requireing more from the PC. Whether this is true or not I don't know, but two days ago it needed a reboot for me to get that solved. But there's more variations to this;

Can you tell, does it interrupt the last track (near the end), or is it right in between of both ?
If the first, then it's the PC. See if a reboot helps and please let me know.

[edward]

Can you tell, does it interrupt the last track (near the end), or is it right in between of both ?

I guess you didn't understand what I meant by:

It appears to happen during the "preprocessing" when it loads the next track.

I am watching Engine3.exe in Task Manager and right when the Memory increases (as it loads the next track) is when it "hiccups". So, it is not right in between both, but rather around 2 seconds before the end of the last track.

But the "click" is right in between both.

[PeterSt]

Ok. But what you don't know is that either could hold up the other.

Did you try the reboot now ? 99,99% chance it helps.
Edit : ... for the hicups !

Were the clicks gone in u-11 and now they are back ?

[edward]

Did you try the reboot now ? 99,99% chance it helps.
... for the hicups !

Too late to try now. I will try again tomorrow. 

Were the clicks gone in u-11 and now they are back ?

Well . . . no I did not notice any clicks with 0.9u-11, but as I said, I was not (could not) using 'Double'.

[PeterSt]

I just realized ... you say that this happens with Double, Right ?

[PeterSt]

but it only all started with 0.9u-7 when you introduced the 'Mem' checkbox.

Edward, don't get psychologically influenced by this. Of course this *was* so, but not anymore.

What happens though is that with Mem unchecked there's pre-processing which wasn't there before. This spikes one of the cores to 100% (you'll see 50% with two cores), and it will be (during) this time that playback gets interrupted.

Remember that I told you that it looked like the system is influenced such that the priorities / time slicing doesn't work properly anymore ? I think I for myself proved a few days ago this is just so.
First the system got "deteriorated" to a level even under Windows 3.11 (I saw that one time before), and buttons almost looked like Dos. Then there was no way the track loading would not interrupt playback. Playback could hold up track loading, but once track loading happened, playback stopped. Single user processing ...

As you know, you already *set* your system (more or less) to behave like that. It is my theory that the processing just takes too much (or takes too long) to let it be unnoticeable.
Remember, the trackloading happens at the lowest priority already, and can't be influenced by you (but the "opposite", the playback thread, you can). If you come to the conclusion that you don't like this net, I can only advise you to check the Mem box. It will then be a tradeoff between more accurate playback because of the coding, versus your stuff that indirectly does the same.
But as said, it's my guess that a reboot helps.

Btw, I just tested and retested ... nothing wrong here with Double as well as normal (used The Wall from Pink Floyd for it).

Lastly, don't hesitate to go against all. We'll work it out.


PS: Might is help you, I'm using Core Scheme 3, and actually never try anything else anymore ...

[RX1]

Remember that I told you that it looked like the system is influenced such that the priorities / time slicing doesn't work properly anymore ? I think I for myself proved a few days ago this is just so.
First the system got "deteriorated" to a level even under Windows 3.11 (I saw that one time before), and buttons almost looked like Dos.

I had the processor running up to almost 100% a couple of times in end of a song, nothing happend exept the windows taskbar changed color from black to white, but I have not been able to either repeat nor isolate what happend - sorry! Could that be kind of same thing as problem you described?

Else I find u12 sounding good so far. I run 16/44,1 and back to upsampling via Purcell to DSD, with Q=0.

[PeterSt]

exept the windows taskbar changed color from black to white [...] Could that be kind of same thing as problem you described?

Oh yes. Those kinf of things happen all the time (but they may stay away just the same).
It can be restored (all becomes nornal), or just stay.

[edward]

Did you try the reboot now ? 99,99% chance it helps.
... for the hicups !

OK Peter, the album I'm playing with is "Spiritchaser" (from Dead Can Dance). I played after a fresh boot, and the same thing happened as before (and in the same spot - so it must not be all that random). Here's what happened:

I started with track 2 and there was a hiccup between 3 & 4 as well as a click - I stopped there.
Then I started with track 3 and there was a click between 4 & 5 and every song after that, and I didn't have a hiccup until between 6 & 7.
Sounds random right? Well, the hiccup seems to be happening at the point when the Memory usage for XXEngine3.exe gets to around 1GB. And the click just seems to happen after the second track and then continues. FWIW, If I start with track 4, then there is no click/hiccup between 4 & 5.

[PeterSt]

Well, at least I have that album !

I started at 2 (Attended), moved the slider towards the end, and had a hicup. Whether that went along with a click I can't tell.
Repeating that wasn't possible, which should lead to the disk access doing it (a second attempt not needing the disk anymore because all is in cache).

Sadly, playing the remainder of the album didn't cause any hicups or anything.

I used the settings from your sig, but Core Appointment Scheme 3.

Well, the hiccup seems to be happening at the point when the Memory usage for XXEngine3.exe gets to around 1GB. And the click just seems to happen after the second track and then continues

Somehow in here is the crux. I now think of the combination with your earlier observation : the memory not being freed at the first couple of tracks. To too, usually is not repeatable. I have been watching this closely since then, and it occurred to me that I am now able to free "fixed" Vista memory, which otherwise won't go. Just overrun the memory (by much consuming tracks), and from then on my Vista uses in the 300 range (which otherwise would be 600).

Ha ! During typing of the above I restarted at the end of 2, let run 3, and 4 began with a click !
Now, indeed (or coincidentally ?) XXEngine3 used 1.1GB of memory (you can't see that in below DCD01 but I checked).
Next this happened (and this too is one which keeps on puzzling me) : I restarted near the end of 3, 4 loaded, the cursor moved, but no sound. This is the occasion that Engine3 just runs, but it beats me where it hangs out (in the code). FWIW this is DCD02 (say, for my own reference).
Then I again repeated this, and then it played without click.

Ok, and this time during typing of the above, from 4 to 5 a click. Total mem usage only just over 1GB.
From 5 to 6 the same (ver small tick).

Well, I have the example ...
Thanks again for your efforts Edward.

Edit : Since I have these subsequent pictures, there's also something visible that I kind of expected : The heading "wisselbestand" meaning swapfile, shows a decrease of 500MB between the first and thye second picture. I expected something like this because we may expect that when the OS manipulates the swap file it gets all the priority there is. Note though that this never will skip coding parts (but the hicup skips sound) because otherwise we will get one big mess (wrong byte order). So what might happen is :

a. The OS thinks more memory is needed than available (might just be true btw);
b. it moves something over to the swap file.
c. This is logic, because at hicup time as I've seen it, there's more IO going on than expected (by me).
d. This is not the best thing in general.

Now, during the playback of 7 the following happened :
The memory had gone up to 1.74GB.
At the move to 8 it went down to 0.96GB.
Playback stopped and even XXHighEnd stopped (cursor stayed at the beginning).

I quickly made a screenshot when 8 was still playing but forgot to save at making a next screenshot of the 9 situation, at which I found that I didn't save the previous one, and then didn't save *that* one.



PS: Bueatiful dry drums in that 3rd track, don't you think ?

[PeterSt]

2.5 hours later ...

At this moment I can come to no other conclusion than that this whole .net / Vista / whateveritisstuff s*cks allover.

I have switched off the virtual memory as fas as possible (Re: Disabling Virtual Memory can cause errors and the result I get from that is that the used memory grows and grows.
So now I know how "Managed" dotnet works for it deciding when to delete memory. The G.d. thing first copies to virtual memory, and then lateron it may delete it (which IMO can happen only after a copy-back to the physical memory first).

[PeterSt]

I hope to have solved it ...
See 0.9u-13.

[PeterSt]

Additionally :

Edward, like I said, your (remarkable) remark about the memory Engine 3 using, indeed gave me the clue;

It appeared to me that all the memory that should be free went right to the paging file. But, as it seemed it stayed into physical memory as well. I think this is about one part of it staying in physical memory and another part was just moved to the paging file. Both should not be so, and the first triggered the last (because that too mr. Bill Gates wanted to preserve).

I'm not sure what in the end triggers the deletion afterall, but as I found earlier, it has to cross some boundary. And, this boundary shouldn't be crossed with a too large amount, or otherwise things can't cope. Think of a "collision", and when *that* happens, we're out of sound.

For those who really want to know, all seems to be related to the thread all happens in. So, within one thread this is out of control. But, dedicate the memory to the particular thread, and it *is* removed immediately when the thread finishes.
So this is how I solved it (and in fact how it emerged as a problem) : do not keep memory over thread boundaries.
Oh, in fact this is no problem for the OS, because it appropiately deals with it, but it's my guess that when this has to be dealt with, the one part has to move out, while the other part has to move in before it can be deleted (that is, this seems very normal to me from a structured means of programming). And that is where it went wrong : there's bits of I/O's back and forth (instead of a sequential stream) and that holds up too much.

Btw, this is (by me) excpected to be kind of similar to the OS performing whatever it is according to I/O, where the disk light goes on and off all the time. It is that what can cause a hiccup during playback. That is, I *never* experienced any hiccup while the disk light was continuesly on. This must be a matter of (too many) interrupts (at the first).

Also, I had the lucky occasion to be able to test things under the "worst" conditions (again, to my interpretation), which was during many services had shutdown (Note that even your network services may shutdown ....

But let me know if it still doesn't work for you.

Peter

[edward]

I have switched off the virtual memory as fas as possible (Re: Disabling Virtual Memory can cause errors and the result I get from that is that the used memory grows and grows.
So now I know how "Managed" dotnet works for it deciding when to delete memory. The G.d. thing first copies to virtual memory, and then lateron it may delete it (which IMO can happen only after a copy-back to the physical memory first).

  
Uhh . . . should I have mentioned that I've been running with Virtual Memory turned off for like the last 9 months with no problems.

Note : By the same process of development it was proven that it is not allowed to shut down the virtual memory, never mind it looks like that can't be achieved it all (it can, but the OS keeps on having some for itself).
When no virtual memory officially is active, you will bump into the limits of the physical memory, recognizeable by some more harsh cracking during the load of the next track (this is not way loud cracking, but sure more nasty than the cracking which may emerge e.g. during a service shutting down).

What do you mean I'm not allowed to shut down virtual memory (or that it can't be achieved)?? I did this a long time ago because of the benefit of better SQ. Is it time for me to re-evaluate this? (And, that crackling thing is interesting.)

Anyway, back to the topic:

But let me know if it still doesn't work for you.

Peter

It still doesn't work. Neither with virtual memory off nor on. I even tried setting "DisablePagingExecutive=1" in the registry. Still get the hiccups. OH! But I think the clicks are gone!

On a side note - the SQ changed between 0.9u-12 and 0.9u-13. Did you expect that? My first impression is that it is "better".



Edit:

Remember, the trackloading happens at the lowest priority already, and can't be influenced by you (but the "opposite", the playback thread, you can).

Might this help - I just tried the same scenario with Thread Priority set to "High" (instead of RealTime) and still get hiccups.