In the very end : no, I don't think so. It would be postponing the culprit. You could run more tracks before it occurs again.
All based upon what I wrote before, and maybe that is wrong by itself ...

Looking at Attended playback and native WAV files :
- When used amount of the memory grows (or just changes), the track is being read from disk (etc.).
- The very moment the time position cursor starts to move the next track is playing.
I think that is all what can be said about things to observe.

Sidenote : your copying from RAM disk will be another cpu intensive (small) task, which is not there when the file is read from disk.
Also : never mind earlier versions working OK. It is good for me as a reference, but it doesn't imply errors as such in the newer version; things are just too different, and the memory used just *is* more ! (somewhere I reasoned how much more it would be opposed to mefore). Of course, this does not happen with the Mem box checked.

Then :

This is more complex than it seems, and this is about the way dot-net (or possible Vista) deals with the memory management.
Also important to keep in mind : when the virtual memory is Off, it just won't work right. Better for you may be : won't work right anymore (?). And please don't forget : the virtual memory just IS NOT Off, but her limit is lower. So, the physical size of the virtual memory is small (2MB in my case) but the behaviour stays exactly the same ! When I tested with this, crackling occurred during the load of a track with a kind of nasty nature. No real hickup, and you could say that 30% of the sound still could be heard.

Another important thing to observe is that as far as I can tell, things are just wrong. Wrong by Vista or wrong by dot-net. You can see it for yourself :
When memory decreases, the virtual memory decreases just the same. When it grows, the virtual memory grows along with it. The correlation with the program (XXEngine3) is kind of hard (even for me), but the correlation between physical memory and virtual memory sure is there. And mind you, we talk in terms of 500MB here that is added or removed from virtual memory (hence disk !) within the snap of your fingers. Btw, it would be logical to state that the "virtual memory" showed, is including the physical memory, but that's not what the screen says ("in swap file"), and furthermore the behaviour is kind of "logic" if one thinks like this :

Physical memory in use is copied immediately to the swap file, but with lower priority. When additional physical memory is needed, all what has to be done is free it (because it's already in the swap file). Smart thinking, and all is faster now. Also try to see the correlation with "dot-net (memory) managed code", where one golden rule is on top of all : It is not you the programmer who can decide to free memory. But don't worry my boy, because all is taken care of.
It is my idea that things go wrong when memory needs to be freed from the swap file. On that matter, try to imagine that this process will have a higher priority than the process copying the data to the swap file. This is kind of logic of course. But maybe this priority (to the sense of stalling other things) isn't even important ...

To me, now, it more looks like a piece of memory (at the array level) is temporarily just not available or so. This then should happen during the process of reorganizing stuff by Vista, and which should be the case at removing in fact already not used memory from the swap file. That is, to me it became clear that the hiccup emerged at the physical memory growing larger than available, which right after that shrunk to normal proporptions. This shrinking is not caused by me (as far as I can tell) but just the notice by Vista that at needing more *that* is the time to physically free what was already freed logically.

As far as the comparison with older versions (or Mem checked) go : keep in mind that before (or with Mem checked) only the 50 MB of a 5 minute track was temporarily needed, whereas now this is 6 or 8 times more (150 - 200 MB). This is significant, *and* better noticeable by us, looking at the task maganer plot.

After I noticed that it all wouldn't work without assigning virtual memory (while all easily would fit in my 2GB of physical memory), I assigned 2 GB of virtual memory to each drive in my system.
Lastly, without my own "memory managing" adjustments, it still wouldn't work. With it does (for me).

Concluded : instead of my before suggestions (last post), you better try to assign virtual memory again. Keep in mind : Vista uses it anyway, so it shouldn't matter for SQ or whatever. But it won't run into limits this way. Thus : when Vista finds that the 2GB of physical memory will be used up, it frees memory, but IMO first whatever the amount is that doesn't fit, goes to virtual memory. Theoretically (and as to what I observed) this is the same amount as the physical memory is (2GB in my case, and which showed after shutting off virtual memory). From this by itself follows that the more than 2GB physical memory needed, will never fit in 2GB of virtual memory, combined with the philosophy (beginning of the post) that *all* physical memory will go into virtual memory as well.

Sorry for this rather unstructured post.
Peter

And Edward, would you be able - and willing - to test this briefly with all your services On ?

In the mean time, when a, say, 5 minute track loads, how long does your disk light lit ? Please keep in mind that you can only test this properly for a first time per track. Otherwise the cache is involved.
Also you could try to notice whether the hiccup occurs during the disk I/O or right after that (during the cpu peaking).

Lastly, you might show a plot of the cpu useage during the track boundaries, but please tell me how long (minutes) the loading track is.

Thanks.
Peter

Right. So now you are telling me that with all the testing of that one particular CD, where I could copy the behaviour, and where now for me the culprits have gone, for you it's not so ? I've been playing that CD the whole day !!
(anyone wants to buy it ?

My remarks about the virtual memory are about when you are near to run out of physical memory. Please keep in mind, I only could solve the issue that was added with some of the before versions. The same issue in the more general code is still there and can't be solved IMO (because I *have* to work with different threads and shared memory).
That in the mean time this doesn't bother you at all, is something else, which may only prove better that I'm dealing with something else afterall.

Now, if you can find a way of making me clear how to copy what you still have ... I can't for the moment.
Btw, I ran at least 6 other CD's yesterday night with the same test setting, and not any hiccup.

Must think ...
But if you can come up with anything ...

  Not Duelund I suppose ...

But be honest ... your camera is too small to capture your ghetto blasters !

Additionally :

Edward, like I said, your (remarkable) remark about the memory Engine 3 using, indeed gave me the clue;

It appeared to me that all the memory that should be free went right to the paging file. But, as it seemed it stayed into physical memory as well. I think this is about one part of it staying in physical memory and another part was just moved to the paging file. Both should not be so, and the first triggered the last (because that too mr. Bill Gates wanted to preserve).

I'm not sure what in the end triggers the deletion afterall, but as I found earlier, it has to cross some boundary. And, this boundary shouldn't be crossed with a too large amount, or otherwise things can't cope. Think of a "collision", and when *that* happens, we're out of sound.

For those who really want to know, all seems to be related to the thread all happens in. So, within one thread this is out of control. But, dedicate the memory to the particular thread, and it *is* removed immediately when the thread finishes.
So this is how I solved it (and in fact how it emerged as a problem) : do not keep memory over thread boundaries.
Oh, in fact this is no problem for the OS, because it appropiately deals with it, but it's my guess that when this has to be dealt with, the one part has to move out, while the other part has to move in before it can be deleted (that is, this seems very normal to me from a structured means of programming). And that is where it went wrong : there's bits of I/O's back and forth (instead of a sequential stream) and that holds up too much.

Btw, this is (by me) excpected to be kind of similar to the OS performing whatever it is according to I/O, where the disk light goes on and off all the time. It is that what can cause a hiccup during playback. That is, I *never* experienced any hiccup while the disk light was continuesly on. This must be a matter of (too many) interrupts (at the first).

Also, I had the lucky occasion to be able to test things under the "worst" conditions (again, to my interpretation), which was during many services had shutdown (Note that even your network services may shutdown ....

But let me know if it still doesn't work for you.

Peter

Ah !, then I can say it works for everybody !
Now go for the Core Appointment !!

And I am sorry it took so long ...

Hahaha Pedal ... you are this first one shdouting this outloud overhere ... Especially mind your last sentence ...

FYI my loudspeaker cable (ok, to mid/high) is 0,75mm2 flat silver wire. I just got it, like it, and have the idea that this (in silk and oil) flat cable avoids the skinning effect.

Btw, when do *you* show your speakers ... at last !!

Engine #3 :

By the time your screen starts to look like below, you may expect your network services to have shut down (which happened to me);
For those who load their data over the network this is important of course.

In that case, higher your Thread Priority one step.
Otherwise the SQ will only get better of it.

I hope to have solved it ...
See 0.9u-13.

It is strongly advised to not use your system without a preamp (or the pre-amp at max volume for that matter) if you or your speakers won't be able to handle a situation that e.g. the file contains wrongly formatted data or otherwise - because of which cracks may emerge with an energy beyond imagination.

The following changes have been applied (all Engine #3) :

• Due to the "preprocessing changes" as emerged in 0.9u-7, additional memory was needed for that. It now appeared that the way this was applied frustrated the memory management dot-net wants to have it. It was found that the "managed" features from dot-net (largely coming down to dot-net deciding when to free memory) are, say, fake. What happens is that memory indeed is freed at a time "good for you" but in the mean time that memory is copied to the paging file (aka swap file / virtual memory). It is this process which (assumedly) the highest priority, that by itself frustrating playback; A bit depening on the "state" Vista is in (and that by itself deteriorates by means of the "force" XX applies -> shutting down services and all) hiccups and clicks could appear.
  It needed some tricks to let the memory management behave differently, but it looks like it is working now.
  
  Note : By the same process of development it was proven that it is not allowed to shut down the virtual memory, never mind it looks like that can't be achieved it all (it can, but the OS keeps on having some for itself).
  When no virtual memory officially is active, you will bump into the limits of the physical memory, recognizeable by some more harsh cracking during the load of the next track (this is not way loud cracking, but sure more nasty than the cracking which may emerge e.g. during a service shutting down).
  
  
  
• The so called "Administrator Message" hopefully does not arise anymore;
  This could happen even though all settings for having the proper Administrator Rights had been applied.
  Because this depends on language stuff it could not be tested. So please let know if it works now, for those where it did not.
  
  
• Even though the "DAC Is" settings implied a DAC that supports 176400Khz at least, Quad was not possible because a message preventing that. This is solved now.
  
  
• People reported they liked the sound from 0.9u-8 better than the versions after that;
  A new checkbox "Ldn", meaning "Less Dynamics" is introduced, and when ticked all should show the same SQ as how it was in 0.9u-8.
  
  Careful here, because this only applied to those situations where the sound got more dynamic from off version 0.9u-7 (!!), and it is not easy to describe where this is (but you could start reading the Release Notes of 0.9u-7).
  Don't get confused by the reference to 0.9u-7 and 0.9u-8 at the same time, which is related to people more (or better) listening to 0.9u-8 for various reasons.
  If we incorporate the Mem checkbox (not properly working in 0.9u-7) and when *that* applies, and include Cue Files which just are not subject to that ... as examples on how to interpret when the Ldn checkbox applies ... we'll see that it will be a placebo if we are not careful. In brief an attempt to point it out :
  
  When the DAC is 16 bits, it will apply always.
  When the Mem checkbox is not checked and the DAC is more than 16 bits *and* 32 bit padding is implied (like the digital volume would do, or just the file being 24 bit would), than too it applies. If it only not is about a Cue File.
  
  Maybe this helps : The chance that it not applies is very small.
  
  Edit : Especially for SeVeReD : Before you ask, yes, for you this checkbox will have its effect.
  
  

Hi Bernard,

Take a PCI card (and not even the cheapest one). It does matter (much) for me, and I encountered the same with others.
Anyway for the 10 euro such a card costs, you can always try !

Peter

2.5 hours later ...

At this moment I can come to no other conclusion than that this whole .net / Vista / whateveritisstuff s*cks allover.

I have switched off the virtual memory as fas as possible (Re: Disabling Virtual Memory can cause errors and the result I get from that is that the used memory grows and grows.
So now I know how "Managed" dotnet works for it deciding when to delete memory. The G.d. thing first copies to virtual memory, and then lateron it may delete it (which IMO can happen only after a copy-back to the physical memory first).

Hmm ... Today I tried to switch off the paging file of myself. It appears to be impossible ...
To me it looks like that the OS will always be using it for itself. "Itself" does not mean XX, because when XX needs it, it errors (out of memory).

Well, at least I have that album !

I started at 2 (Attended), moved the slider towards the end, and had a hicup. Whether that went along with a click I can't tell.
Repeating that wasn't possible, which should lead to the disk access doing it (a second attempt not needing the disk anymore because all is in cache).

Sadly, playing the remainder of the album didn't cause any hicups or anything.

I used the settings from your sig, but Core Appointment Scheme 3.


Somehow in here is the crux. I now think of the combination with your earlier observation : the memory not being freed at the first couple of tracks. To too, usually is not repeatable. I have been watching this closely since then, and it occurred to me that I am now able to free "fixed" Vista memory, which otherwise won't go. Just overrun the memory (by much consuming tracks), and from then on my Vista uses in the 300 range (which otherwise would be 600).

Ha ! During typing of the above I restarted at the end of 2, let run 3, and 4 began with a click !
Now, indeed (or coincidentally ?) XXEngine3 used 1.1GB of memory (you can't see that in below DCD01 but I checked).
Next this happened (and this too is one which keeps on puzzling me) : I restarted near the end of 3, 4 loaded, the cursor moved, but no sound. This is the occasion that Engine3 just runs, but it beats me where it hangs out (in the code). FWIW this is DCD02 (say, for my own reference).
Then I again repeated this, and then it played without click.

Ok, and this time during typing of the above, from 4 to 5 a click. Total mem usage only just over 1GB.
From 5 to 6 the same (ver small tick).

Well, I have the example ...
Thanks again for your efforts Edward.

Edit : Since I have these subsequent pictures, there's also something visible that I kind of expected : The heading "wisselbestand" meaning swapfile, shows a decrease of 500MB between the first and thye second picture. I expected something like this because we may expect that when the OS manipulates the swap file it gets all the priority there is. Note though that this never will skip coding parts (but the hicup skips sound) because otherwise we will get one big mess (wrong byte order). So what might happen is :

a. The OS thinks more memory is needed than available (might just be true btw);
b. it moves something over to the swap file.
c. This is logic, because at hicup time as I've seen it, there's more IO going on than expected (by me).
d. This is not the best thing in general.

Now, during the playback of 7 the following happened :
The memory had gone up to 1.74GB.
At the move to 8 it went down to 0.96GB.
Playback stopped and even XXHighEnd stopped (cursor stayed at the beginning).

I quickly made a screenshot when 8 was still playing but forgot to save at making a next screenshot of the 9 situation, at which I found that I didn't save the previous one, and then didn't save *that* one.



PS: Bueatiful dry drums in that 3rd track, don't you think ?