Shut off Virtual Memory

[astacus21]

Also a good idea for those who have enough memory is to disable the virtual memory.

Windows have a built in function that stores some data to c:\pagefile.sys (hidden file) and load them to memory, as soon as you run the program that those data belongs.
You can disable this from my computer (right click)> properties> advanced system settings > advanced tab.
Here in the performance area click settings and in the advanced tab you can find the virtual memory.

Click change and set it to no paging file.

[PeterSt]

Also a good idea for those who have enough memory is to disable the virtual memory.

I don't think this is true for Vista;
First of all, this just CANNOT be shut off, no matter you think you did. The OS will just keep on doing it for herself, which you can see in TaskManager. However, if you can prove (!) that for your system this can actually work, then yes.

Secondly, you will receive unexpected errrors (hence errors which otherwise don't appear), especially when you indeed are able to really shut it down (see above).
Also, please note that at certain settings and track lengths you just *will* run out of internal memory, or can't control (= expect how much) that decently, which is subject to the way XX works.

All 'n all it is certainly *not* my advice to shut down virtual memory. Otoh, anyone who feels comfortable with that, including what he can expect hence interpret the errors from it ... go ahead.

[astacus21]

Hi peter

The virtual memory of windows (vista,xp etc..) is a mix of physical memory (paged physical memory) and hard disk space
(pagefile.sys).
With the above procedure you disable the part of virtual memory, that page data to the disk. After this you can also delete pagefile.sys from your disk because it is not in use.

You cant disable the entire virtual memory, it is not allowed by the OS.
I'm working with the above setting for almost 2 years and until now i haven't saw any error on screen.
My physical ram is 4gb, but i have also try with 2gb and i get the same results.
Some programs like adobe audition can show a warning that virtual memory is disabled, but you don't have to bother about it.
This procedure cannot create Blue screen or something bad to your os.

[PeterSt]

With the above procedure you disable the part of virtual memory, that page data to the disk

You cant disable the entire virtual memory, it is not allowed by the OS.

I sure don't want to catch you on wording , but you seem to think the swapping is going somewhere else (hence not to disk) ?

My physical ram is 4gb, but i have also try with 2gb and i get the same results.

Why should you try 2GB ? the 4GB is much better of course (and note you can address that in W2008), but there's a calculation somewhere in here that it really is not (it was 9GB I think) when you only play the high samplerate and high bitrate files at the best quality. But watch out : this is not related to the swapfile thing at all, because with or without, XX will die when too much memory is needed. It just can't use the swapfile. However, other processes will need the memory at the same time XX eats it (almost), and that is where you'll get the unexpected errors.

But as said, when you are comfortable with the settings as you explained, nothing stops you. I just don't want to explain the obvious to those who have the errors.


Just for your reference : Re: Track length (remember, unrelated to the virtual memory, but indicating that your 4GB sure isn't enough some times ).

[PeterSt]

PS: Another thing is, that the page file really isn't used when all of the above is applied;
If you'd stroll through this topic : 0.9u-12 --> Hiccups and Clicks, you can derive from there that it sure was before (and I bet this was about the prefetching idiocy). Especially see this post from that topic.

The fact that the page file isn't used anymore can easily be checked with the Source Control program. During playback with XXHighEnd that is.

[astacus21]

For sure when we have to deal with big files then it is not recommended, because obviously it will cause xxengine to close BUT
when we use the regular 44.1/16 wav files, a 1,4 GB memory is enough for about 2 hours playback.

...swapping is going somewhere else (hence not to disk) ?

I don't mean that the swap file goes somewhere else, i think you haven't understand what i'm trying to say.

The swap file is not exactly the same thing with virtual memory. It is just a part of VM.

Virtual memory is a memory space for the os, that it constist of a space in physical memory (the size is controlled by the kernel of os) and
the swap file, a space on the hard disk. By disabling the swap file, you just allow system to page memory to the physical memory only.

When a program needs this data from the page table, a chip insude cpu the mmu (memory managment unit) "sends" a request for this data to move from the vm, to the physical one. So if this data is allready to the physical memory part of vm (if you disable swap file this happens all the time) , then the os just translate those virtual addresses to physical memory addresses. This translation is much faster with swap disabled.

I know all this brings headache, but just try it and if you don't like it, is easy to set it back.

[PeterSt]

when we use the regular 44.1/16 wav files, a 1,4 GB memory is enough for about 2 hours playback.

I am sorry, but you really should dig up the posts that present the calculation to you.
For your two hours this would be 4.2GB plus the, say, 600 MB the OS (Vista) needs.

About the other stuff, you might call "not physical memory" virtual memory not being on the hdd, but unless it's on the CPU itself I wouldn't know where it is. Of course, you say it is in physical memory itself, but the only conclusion I draw from that when it is true, is that this is stupid dealing with memory (management). Just a waste. That is, I wouldn't know (nor design it) why to copy a part of memory to another part of memory, sitting there and waiting for when it's asked for. As said, unless it's on the CPU, although that's really the other way around (because that's fast memory (which we tend to call a cache)).




PS: About the 2GB limit and what happens then, I tend to believe. There is no way though that I would call that virtual memory.

[astacus21]

OK peter please delete the posts,are unrelated to the subject. But i  still believe that you haven't understand...
Please read carefully. It 's no so difficult anyway..

[PeterSt]

But i  still believe that you haven't understand...

That is true for sure !
I will dive into it seriously. Thanks.

[PeterSt]

Ok, I think (!) what you refer to is mapped memory address space. Thus :

IMHO there is no such thing as virtual memory outside of the paging file, and what remains is physical memory that can't be addressed in one 32bit address space. This is the "mapping" onto other address spaces. With 64bit things are different, but it still depends on drivers and mobo's how to deal with that.
Another thing, but in the same area, is how the memory is appointed for 32bit systems (for 64 bit this is no problem). I mean, when you have e.g. 2 GB of memory, only with the best setup (again drivers/mobo) that 2GB can be available to you. Take my own system, which has 2GB but only 1.35GB is available to me (and that 1.35GB includes the OS). Now the hard part : my "virtual memory" (excl. page file) is 1.94GB. Again IMHO the only thing what happens here is that drivers etc. are not able to appoint the full memory of 1.94GB, and the part outside of the 1.35GB is addressed via memory mapping.

Lastly, there's also the mapped area for driver space, and this is the amount that is calculated from the total amount of physical memory minus the total amount of virtual memory. In my case this is 2048 - 1940 = 108MB.

Here's all in a row :

Total amount of Physical memory : 2048 MB
Available Physical memory : 1350 MB
Total Virtual memory : 1940 MB
Available Virtual memory : 1280 MB
Space for swap file : 0MB.

So if I may, yes, outside the swap file there's virtual memory, and this is more than the officially available physical memory. But this is caused by, say, flawing drivers or a mobo which can't cope.
Note that this is similar (but more commonly known) to installing 4GB in a Vista consumer 32 bit (OS) machine, while only 3.1GB of it is available. In a 64 bit (OS) machine, the full 4GB will be available, but there things are (seem to me) a bit upside down : that may show 4.5GB, the 500MB is related to driver space again, but something is here really "virtual" and it is not there (something with dead space -> nog used anymore, but it still can be obtained, blahblah).


Now, to finish it off a bit for this post (and my side ), there's the swap file hence paging (please check this yourself with Taskmanager - Performance) (btw. this is 32bit Vista);

When I allocate a paging file of 2GB, "paging" is allocated of 4GB. Don't ask me where the additional 2GB come from and go to;
When I do not allocate a paging file, "paging" is allocated of 2GB.

In either case the additional paging space (hence the space I don't ask for), is almost equal to my total amount of Virtual memory, namely 1981MB (Virtual memory is 1940 MB in my case, see above).
What I have seen (referring to the topic I referred to earlier), is that the space consumed in that additional paging area is equal to what I stuff into memory itself. I explained that (just my own thoughts) as : "this is how Vista works, and when it has to swap out memory, it's already done ... aint that smart". The priority at doing this is obviously low (the stuffing of the real memory goes first), but it causes additional IO anyway.

... and now I am going to give you right ... Just readon on ...

What is IN USE by that paging mechanism, is the amount of (shown) physical memory plus the amount of driver space (that is, it shows 100 MB more than used physical memory, which is nearly equal to my 2048 - 1940 v.m.).

Then, interestingly enough, the space consumed by that paging area (the one we didn't appoint) is that when I load 100 MB additionally into physical memory, that space grows by 200MB.

So how does it work ?

The line "Page file" (see picture below) shows the total amount of virtual memory. In my case the page file has been shut off, so you see (the right figure) my total amount of Virtual memory (or very close to that) and which is the available physical memory plus the mapped address space.
The left figure - which shows the total amount being in the "page file", actuall is the total amount of physical memory I use. This is the shown amount in green (not shown in the picture below) plus ... the amount used in the mapped address space which in fact adds to virtual memory.

The above can be desribed much more easy : The amount of physical memory which can't be addressed on one go by your system just *can* be used as if it were a real page file on disk. And yes, as you already said, copying from there goes must faster.
But also : Like when the normal paging file is active, EVERYTING is copied to that area, while Vista is so smart to do that in advance in case I need it. One small downside : When that space is full, the program needing it dies.

And the latter is the phenomenon I talk about (also in that other topic) : I cannot control what's kept in there and what not, and coincidental "collisions" of really needed space with the OS just adding to that additional virtual area, let grow my system out of memory. Mind you, this is not true at all, because this is not swapped out data, but prefetch data (hence is copied there in case it needs to swap out from the real available memory area).

The above can easily watched, by knowing a bit what (e.g. XX) really needs, looking at the green figures witch match my idea about it, that showing e.g. 1000MB, while in the mean time the Page File number (left figure) grows to over 1500MB. The calculation to this is rather consistent : just look how much memory is needed (green figure) when no user programs are loaded.Say this is 500MB. The Swap file in use will be 0MB here. Now, when you load 500MB of user programs, the green figure shows 1000MB and the Swap file will show 1500MB. Load another 250MB and the Swap file (remember, this is just physical memory) wants to have 2000MB which is not there. It is here when things get stressed. As a not unimportant side note counts that when my user memory decreases, the Swap file decreases as well. This is logic, because it cannot be so that I would need its contents anymore.

So, in that other topic I proved that things go stressed within Vista when that Paging amount reaches its limit, and it must give priority to free it; it stalls the other processes (briefly), including sound.
Besides this we might wonder what happens when the user memory in use needs to swap out; remember, normally the OS can depend on the copy already being in Virtual memory, but not now. Now it has to make the copy in advance, and I dare to put some horses on it even can't.

It is nice that Vista can make this copy in additional Virtual (non hdd) memory for me, as a catch to my system not being able to have the physical memory fully available in the first place. But you tell me : what if my available physical memory would have been not 1350MB but 1700MB. There would only be some 300MB left for the additional Virtual memory, and things get stressed all the time. Keep in mind : this is not memory that contains swapped out user space ... this is a prefetched copy of it. If the prefetched copy gets swapped out because the area is full and cannot contain the full user memory, the user memory can't get fuller than in my case 1350MB, unless the part that needs swapping out at that moment, coincidentally is in the copy in the additional virtual memory.

I think I said in that other topic : this s*cks all over. Say, out of decent control.
I hope you can agree with me that this is nothing for a normal user to understands, who just *will* receive out of memory errors, and who obeyed my calculation of how much memory XXHighEnd uses and when. This user bought 2GB of internal memory, and doesn't even know that he has 1.3GB available really, which only in the very coindicental best circumstances can grow to around 1.9GB.

Add to this that XXHighEnd runs under Managed Code (which dot-net is), and that there's really no control of when what memory is freed (the OS "manages" this for you). With stupid tricks you can force some things, but not all, or they are to vague to control really.

If you disagree ir have other opinions, I'd be happy to hear them.
Peter

[astacus21]

Peter it's a little bit difficult to me to explain, cause english is not my native language but i 'll try.

It is nice that you make expiriments with memory, as your program, use it in a way that performance is very critical.
It 's impossible to me to answer to the above comments, because we 'll get lost in details.
You can't just look in the task manager properties, to see what the memory management of windows have done with memory, because you see just numbers, but you don't know from where (which application) this data comes.

It is more easy for me to explain, how the os acts, when an application requires memory from the system.

All this conversation is important, if we use a 32 bit os (it doens't matter which one). At 64bit we don't have remarkable limitations (we have but, the limitations come from hardware).

Let' s take the following example, that we have a system with 32 bit processor and 4gb of physical memory. I use this example, cause it contains all the others circumstances like 2gb etc...

A 32-bit processor uses 32 bits, to refer to the location of each byte of memory. This means 2^32, about 4.2 billion unique locations. Let's say 4 GB. Those are all the possible addresses of memory, that the cpu can use. So above this number the cpu just can't see those locations and can't use them.

Let's go now to the OS side, in the above example.
It is built in a way, that every single application that request memory, has it's own VIRTUAL 3gb memory space.
This means, that every application acts like there is ALWAYS* 3 gb of (VM) memory to occupy, independently from the fact that for sure this space doesn't exist in physical memory. So this application starts to PAGE it's data, in a continious virtual space, until it get's all the data to that space.

*The only limitation to this come from the OS, which have the "central'' management of memory and can stop this procedure, (paging) if the application allocates so much space. OS manages all applications paging procedure, in a way that the available PHYSICAL memory & the SWAP file doesn't run out of space.

This can happen, when the os prefetching the most used programms OR when an application requires much space in physical memory like xxengine process.

The 4 gb of our example are devided by the os, to 2gb kernel mode space (os kernel, bios, shadow memory, agp aperure size, etc...) and 2 gb for ALL applications space. (this can be changed with some  tricks to 1/3).

As i said to another post this virtual memory is a mix of space in physical memory and hard disk. If an application requests the vm data,
then a procedure take place and this addresses are translated to physical memory addresses, so that the application CAN USE them.
While this data are not translated into physical address space, only OS knows about them and no application can use them.
So for this translation if we have the swap file disabled, this operation takes much shorter because the hard disk doesn't participate.
This is the benefit.

So if a process from one application that is not in use , allocates about 1gb in virtual memory and you start a 1,8gb wav (1+1,8=2,8 that is bigger than 2gb max for apps) the os will translate, this 1,8gb to physical memory and at the some time it will release the data of the old application so that this 1,8 will fit.For sure this requires cpu assistance and will create clicks and pops.
 
On the above paragraph i said that 2gb can be translated in the best case, because in 32 bit windows this physical space of memory is the maximum that all the applications (user mode) can allocate.


I hope that the above are clear to understand because it is not so easy to explain.


PS: You will never come up with critical errors for OS, because when the physical memory running low, the operating system will release some space for the process that need this space. Don't forget that kernel modules will always have availiable memory in the 2gb kernel space.

 

[PeterSt]

cause english is not my native language

Here's another one !

It is more easy for me to explain, how the os acts, when an application requires memory from the system.

I don't know how you get this information (which I must just trust btw), but it is not said that what you know of it, is working correctly.
But before I get into your explanation, and before we have a misunderstanding, would you be so kind to choose from the below ?
In all cases the Swap file has been shut off ! All cases are 32 bit OS.

a. I installed 4GB of internal memory; when a program asks for e.g. 5GB, it can still run.
b. I installed 4GB of internal memory; when several programs together require e.g. 5GB, they can still run; maybe not all at the same time.
c. I installed 4GB of internal memory; if one - or all the programs together require more than 4 GB, an error is thrown.
d. I installed 4GB of internal memory; if one or all programs require more than the by the OS reported available physical memory, an error is thrown.
e. I installed 4GB of internal memory; if one or all programs require more than the by the OS reported available physical memory, an error is thrown when the reported 4GB limit of virtual memory is exceeded.
f. I installed 4GB of internal memory; if one or all programs require more than the by the OS reported available virtual memory, an error is thrown.

I have the hunch you will choose another option than I will.
I don't think it will be easy to choose (for me the same, because of the way I put it), and if you have the idea a comment has to come along with it, please do so (short if possible).

If you don't like it, never mind !

[astacus21]

The way the os acts is explained in microsofts technical papers. you can search for more info at microsoft technet.

For your a,b,c questions i'm not sure for the answer.But i think it will fail. Everythings it is going to work, if the availiable virtual memory (inside physical), is enough to fit the size of the process. I 'll make some experiment to see where is the limitation to me, as i use 4gb memory without swap.
For d,e,f i think that you sould get an error.

I think that everything is gonna work if it is below 2gb.

[astacus21]

Hi Peter

I make some tests with wav files and the biggest file i was able to play was about 1,8gb, which is near the limit of 2gb that is allowed for applications. It was a single wav about 170 min at 44.1/16 bit. This was with swap deleted.
After that i open some other applications so that the virtual memory get full with data and i try again with this 1,8gb wav. Guess what???
It' works so the procedure that i describe in the above posts, it works.

I've tried also with 2.4gb wav which failed.

After that i set back the swap file at 6gb of disk space and i restart the system so that the new pagefile was set to this size. Then i try to play the same wav 2.4gb but it still fails even at server 2008 x64. It was a surprise for me but after a while i remembered that when you start xxhe in 64 bit, it plays in an emulation mode of x32. So in this way you cant play anything above 2gb.

PS: I have said something wrong in the above posts. In this part that i say about a 2,5gb wav. So i 've made the corrections.
The right is that you cant have over 2 gb proccess in 32 bit. so if you have virtual memory filled with data from other apps and you try to play a file smaller than 2gb then this procedure take place.


**edit : i've found 2 links from MS that are related with the 2gb process:
for vista       http://msdn.microsoft.com/en-us/library/aa906211.aspx
for xp/2003   http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx

[PeterSt]

Ok, so you have been busy !

The way the os acts is explained in microsofts technical papers. you can search for more info at microsoft technet.

Yes, before I posted my large post yesterday I read into it all, and my post did not (try to) disobey that.
Btw, note that half of my post was about reasoning what TaskManager actually shows under "Swap file" (or Paging file, I translated it from Dutch), with the conclusion that that should read as Virtual Memory (in or excluding the Swap file, depening on whether it has been switched on or not). So, my earlier statement that Vista always keeps on using some swap file was wrong. Instead, it always keeps on paging to another part of memory (which is what you said from the beginning), but this is so because the OS can't directly use the full available physical memory.

The latter is my finding, or better : conclusion from everything in the white papers, because this is not described directly. Both "processes" -> 1. not being able to use the full memory and 2. paging to the part which is available - are described separately, and not in the combination I present : by kind of accident the memory which can't be used, can be used afterall because it can be used as additional virtual memory.

What you describe as "the kernel determines its size" (similar), is therefore only partly true. What will happen is that the kernel is able to determine unused memory space, and dedicate as paging area. Also note that not in all cases the memory can be fully utilized, which again is a matter of (wrongish) drivers.



Now, correct me if I'm wrong, but from your orginal subject we both seem to have our own subject. And oh, they are both as important I think.
Thus, your subject seems to be the explanation of how the OS deals with available physical memory vs. not available physical memory but which can extend to virtual memory ...
while my subject is the stupidity of that, and that it can't work (the other half of the large post from yesterday tries to explain that).

Well, I don't blame anyone who can't understand what I have written in that post, but, you can't counteract that with explaining the theories of operation. And please keep in mind : I knew them by the time I wrote that post (which was not so day before yesterday).
Important for you is that we agree, or at least that is how I see it. However, we only agree about the theory of operation, and not about the flaws following from that (remember, you didn't go into my subjects, and explained how it works instead). So :

So what I thought of it before (the topic I referred to) didn't change a single bit now knowing how it works, and you could say that I can see how things work without knowing it really.
Of course one thing changed : you *can* shut off paging to disk, hence no disk IO will occur after that for this reason.

Things may look trivial (the theory vs. the workout in practice), but it really is not IMO;
When you said that obviously the CPU is involved when the additional (!) virtual memory gets full, I claim that what happens there is b*llsh*t, and the CPU activity there is unnecessary and besides that *has* to work with the highest priority because of the processes happening.

The main subject to the latter, is the fact that it is not the normally available physical memory which gets full, but it is the paging area which gets full. And the most important is that this gets full not because of swapped out data, but because of prefetched data in case something will swap out.
I know, you said something like "you can't know what's in that area", but please trust me, I do. -> each byte which is user data which is necessary in the normal physical memory is copied to that additional memory as well (low priority) just in case the normal physical memory gets full.
Again, with some time this can be seen rather easily looking at taskmanager.

This all is rather unrelated to any theory of operations, because this happens with the normal swap file just the same. That is, this is what I expect but cannot see without additional investigation. So, what I expect is that when the additional virtual memory gets full, the swap file is in order, and any additionally needed memory in the paging area then goes to the swap file.
This means - in that case - that each user byte loaded goes to the swap file as well and takes I/Os. Also, as soon as that user byte is freed, again I/Os emerge to free that byte from the swap file. Now :

Since I now know better what happens when the swap file has been shut off, there's a kind of interesting conclusion (until proven wrong) :

As I told in the long post from yesterday, the 700MB or so which can't be utilized in my system (and which is so because of driver/mobo issues), actually is an advantage. Huh ?
Yes, I think so; Where my system starts off with 600MB used memory by the OS (could be 500MB depening on things) I have 700MB spare in there. Now, coincidentally, I have another 700MB of additional virtual memory, so both balance out. And keep in mind : each user byte in normal available physical memory goes to the additional virtual memory as well and I have 700 MB of memory for playing music without the stress of the additional virtual memory getting full and the operations to solve that. Note that in best circumstances (Mem box) this allows me to play to subsequent "full CD tracks" because I actually have 1400MB free, and they just can be utilized. However, when the second track is loaded, playback will be interrupted briefly near the end of track 1.

What I now may claim (must think about it further somehwat) is that the less additional virtual memory you have, the earlier stress time applies. Also, - and this might be the most important conclusion when it's true - when the swap file is just active, stress time does not occur (ok, unless the set limit of the swap file is reached).

astacus21, in order to judge these phenomena you get nowhere at proving the theory of operations to be right (or wrong for that matter); Instead you must do what I did, and just start playing tracks keeping in mind that one minute 44.1/16 takes around 10MB, and watch and watch and watch (TaskManager). You will notice that the green figure (the normal available physical memory actually in use) is not consistent with your thinking, which is caused by the Managed Code phenomenon, and you will also see that the derival of the green figure, the total virtual memory in use grows/shrinks linearly with the green figure, starting off with no user programs loaded (the, say, 600MB). Now, two happenings mat break the linearity :

1. Normal available physical memory gets full
2. Additional virtual memory gets full.

Ad 1.
Will cause anomalies which the OS is able to recover from (stress time).

Ad 2.
No problem, because the data is available in the paging area, which is in additional virtual memory.
Watchout though when 1. above happened first, and I claim an Out of Memory then.

Of further importance at judging the theory of operation vs. practice, is that this all is not about one single program (similar to your 2GB tests of last night), that it also is not about two programs, but that it is about one program dynamically allocating and freeing (huge amounts of) memory all the time, and that the OS takes care of the freeing, and one of the *reasons* to do that is reaching memory limits.
Looking closely, you can see that (somehow) the OS is not "smart" right after XX started playing (better : right after a reboot, and this *always* has bugged me no matter what kind of sound engine I used, no matter XP or Vista); Just start playing, and see the green figure grow and grow, until it reaches the physical limit (my 1350MB or so) and then it drops back. After that happened once, smartness pops in, and everything becomes predictable.

Side note : so, so many users (an I am one of them) reported (let alone those who did not report it) that things go wrong somewhere somehow at track 4 or 5 or 6 at a first playing session. In all cases there is no explanation (by me). But, I *know* the hard way of the OS needing to get rid of obsolete memory is in order there ...

Back to the case, I think I dare to say that I have found the solution hence differences with my system never having problems (swap file On) and a system like e.g. from the topic I pointed to (0.9u-12 --> Hiccups and Clicks and where Edward should try to set the Swap file ON.

Then there is this one :

Earlier in this topic I claimed that with XXHighEnd Audio playback, and having applied the settings as described in How I tweaked my Vista virtually dead, the Swap file will not be used (and where we by now have seen that it will occasionally be used at stress time situations). However :
I think this (Swap file not used) only applies for those who have an amount of additional virtual memory that is at least as large as the normal availabe physical memory, hence who have poor systems like mine. Hahaha. Thus, when you have far less (relative) space in the additional virtual mamory area, the limit in there is reached (long) before the normal available physical memory is full, and the Swap file *will* be used. Oh man, so where this encourages for shutting it off, the problems get only worse because of stress time.


astacus21, I realize that you all can't test this so easily with your 4GB of memory, and I even don't know how much of that additional virtual memory you have. Note though that if this is a lousy 700MB (which is lousy opposed to the 4GB) you hardly can encounter problems there, just because I don't with my 700MB in there.

The conclusions you draw to the 2GB and "one process" seem a bit dangerous to me, because I (so far) didn't bump into anything that descibed "one process". For your (further) mindsetting, think of this :

One 170 minute WAV (must be a hell of a bootleg btw) is not something we really need to play. But, a 60 minute 96/24 (or higher sample rate) is, and by heart this would be 1800MB. I can assure you that this 1800MB must be treated by one single process. But :
In order to get playback going, we must assume that this is needed twice the very least. In this case, this is dealt with by two threads, which I would call processes on this matter. But would they be to this respect ?

Sidenote : I plan to cut large (byte) tracks in pieces when needed, so actually it won't be a problem.

It was a surprise for me but after a while i remembered that when you start xxhe in 64 bit, it plays in an emulation mode of x32. So in this way you cant play anything above 2gb.

Yeah, that's a good one. I would never have thought of that. But be careful :
I can't look into it right now, but I am fairly sure that the process playing the role here (which would be XXEngine3) is *not* compiled for x86. I will let you know later.


Well, this was my typing for the day.

[astacus21]

WOW so many things to talk about, and it is impossible to answer everything.
I'll use some quotes just to make it easier to read. Consider that i'm not trying to prove anything like i know beter than you, or something else.
In the past, i had the some questions and i used the MS knowledge base to find answers.

Let's start, but before this, if you think that something of the below is wrong please ask for it. It is easier for me to answer for specific subjects and no for general things like how the os reacts.
 

half of my post was about reasoning what TaskManager actually shows under "Swap file" (or Paging file, I translated it from Dutch), with the conclusion that that should read as Virtual Memory

I disagree again with the process you describe to find out what it is inside virual memory. I insist that you can't be able to know, what virtual memory have as data, especially after some time (10 min) after boot that the applications have allready occupy the space of vm. Don't forget that most of the people who uses the xxhe, have also many other programs installed that requires some of the space in vm.
Of course if you have a clean installation and you modify some services, then you can say that you leave free space for xxhe but this needs many changes and it is not the default situation.

my earlier statement that Vista always keeps on using some swap file was wrong. Instead, it always keeps on paging to another part of memory.

For this statement and some others i started to explain how the os reacts because it was very important to understand the difference between swap file and virtual memory. As you can see from the taskmanager if you disable swap file you still have virtual memory. So i was trying to explain what is the role of vm.

but this is so because the OS can't directly use the full available physical memory

Another thing to disangree.
Of course os can directly use full of the AVAILIABLE memory.
If you mean that this availiable memory is smaller than the installed on the system is wrong cause you don't consider something else. Some hardware devices like motherboard, vga, network card, soundcards etc...
needs some space on physical memory to operate. this space is given from the system even before the os boots.
Lets say for example that you have a vga card with 512 mb memory. this memory is installed on vga's pcb BUT it requires the SAME space availiable to os memory to cooperate. For this reason MS decided to reduce the availiable physical memory to 3,12gb in a 4gb system, so that there will be no problem for all this devices, that could cause BSOD. (blue screeens).

What you describe as "the kernel determines its size" (similar), is therefore only partly true. What will happen is that the kernel is able to determine unused memory space, and dedicate as paging area.

When i said about the kernel of os, i speak about the memory management part of windows that is very complicated to understand how it works.To understand this we need very specialized knowledge.

Also note that not in all cases the memory can be fully utilized, which again is a matter of (wrongish) drivers.

This is also wrong, as i said before it has nothing to do with drivers, it has to do with the devices that are attached on the system.

Now, correct me if I'm wrong, but from your orginal subject we both seem to have our own subject. And oh, they are both as important I think.
Thus, your subject seems to be the explanation of how the OS deals with available physical memory vs. not available physical memory but which can extend to virtual memory ...
while my subject is the stupidity of that, and that it can't work (the other half of the large post from yesterday tries to explain that).

No reason to disangree with the first, but i told you the reasons why it was better to explain how it works.
I talk for availiable physical memory because when this come close to its low limits then we have this procedure.
I think that the only change to the above that i could make is to delete the word stupidity and write sophisticated, for the way the os acts. If it wasn't worked that way then everytime that a application asks for much memory, then we 'll have a crash of the system.

Of course one thing changed : you *can* shut off paging to disk, hence no disk IO will occur after that for this reason.

OK but this is the reason that start all this conversation after my post. And of course to prove what is the benefit for it.

When you said that obviously the CPU is involved when the additional (!) virtual memory gets full, I claim that what happens there is b*llsh*t, and the CPU activity there is unnecessary and besides that *has* to work with the highest priority because of the processes happening.

when the system translate the addresses from vm to physical, one part of cpu cooperates and it is unavoidable. This is called mmu. I dont' know how fast is this but it can creates click and pops.

This all is rather unrelated to any theory of operations, because this happens with the normal swap file just the same. That is, this is what I expect but cannot see without additional investigation. So, what I expect is that when the additional virtual memory gets full, the swap file is in order, and any additionally needed memory in the paging area then goes to the swap file.

i don't know if i understand the above, but with swap file enabled, while we have paging for an application some data goes to physical memory and some others to swap file. This is controlled by the central memory managment of windows. It has the role to predict which data are going to be used soon and send them to physical memory space, as virtual memory, for faster processing when it transalte them to physical addresses.

I know, you said something like "you can't know what's in that area", but please trust me, I do. -> each byte which is user data which is necessary in the normal physical memory is copied to that additional memory as well (low priority) just in case the normal physical memory gets full.

But how do you know what is the priority that is given for apps, by the memory management. You can't control that. Plus it is a ramdom procedure. You can only control the priority given by the cpu for a process.

As I told in the long post from yesterday, the 700MB or so which can't be utilized in my system (and which is so because of driver/mobo issues), actually is an advantage

It is wrong to say, that they can't be utilized. Os works in a way that it leaves them free, so that it has the space (in user mode apps) to translate virtual memory to physical. As i said before this is not stupid, but the only way to utilize memory without getting blue screens, when the physical memory get close to it's limiits.

Yes, I think so; Where my system starts off with 600MB used memory by the OS (could be 500MB depening on things) I have 700MB spare in there.
Now, coincidentally, I have another 700MB of additional virtual memory, so both balance out. And keep in mind : each user byte in normal available physical memory goes to the additional virtual memory as well and I have 700 MB of memory for playing music without the stress of the additional virtual memory getting full and the operations to solve that

Right if no application is running you can have all this space and more, until 2gb for a system with 4 gb physical memory.

is that the less additional virtual memory you have, the earlier stress time applies.
when the swap file is just active, stress time does not occur (ok, unless the set limit of the swap file is reached).

the right is the opposite.
The less physical you have available the earlier stress time applies, because the translate to physical memory take place when it is going to finish.

... in order to judge these phenomena you get nowhere at proving the theory of operations to be right (or wrong for that matter); Instead you must do what I did, and just start playing tracks keeping in mind that one minute 44.1/16 takes around 10MB, and watch and watch and watch (TaskManager).

I don't know for me it is more important to understand how it works, and not to try to guess. As a matter of fact i 've try it and i don't get clicks at playback.

Now, two happenings mat break the linearity :

1. Normal available physical memory gets full
2. Additional virtual memory gets full.

both of them are imposible to get full. If there is no much space then a proess just don't start!

Looking closely, you can see that (somehow) the OS is not "smart" right after XX started playing (better : right after a reboot, and this *always* has bugged me no matter what kind of sound engine I used, no matter XP or Vista);

Os have to take care of other processes as well. If for example a defragment starts, it needs both virtual memory and physical space that it has to remove it from the upper limit of 2 gb for user mode apps.

Side note : so, so many users (an I am one of them) reported (let alone those who did not report it) that things go wrong somewhere somehow at track 4 or 5 or 6 at a first playing session. In all cases there is no explanation (by me). But, I *know* the hard way of the OS needing to get rid of obsolete memory is in order there ...

So this is the time that memory managment of windows starts to change the data of physical memory. this needs cpu resources.

However :
I think this (Swap file not used) only applies for those who have an amount of additional virtual memory that is at least as large as the normal availabe physical memory, hence who have poor systems like mine.

This is true, and i recommend to disable the swap file only if there is 2gb or more. But think smt else. With 4 gb memory and SWAP OFF i can play about 1,8 gb (170 min) without clicks. with 2 gb which means 1gb for system and 1 for apps, for sure you can play about 90 min of 44.1/16 without clicks in a system that have modified processes with the above tweaks. 90 min is not so little. When i say without clicks i mean for systems that don't have latency problems, because this is another story.
When we talk about 96/24 i think, that only if the program runs in native x64 can handle this situation and if we have enough physical memory 4gb or more.

I need more than a coffe right now 

[PeterSt]

Hahaha, not that I don't like the discussion, but we are running in circles. And I must honestly say, if you keep on applying the theories (remember, the elements of them !) we'll get nowhere really. Besides that, we may be better off if you -no matter it's hard- read what I say, and reject that explicitly (meaning : with arguments) instead of repeating the theories. I presented arguments enough I think, and they can easily be seen hence proved. But :

If you keep on saying that I don't know what is where, well, there's not much to discuss anymore;
If I say that I know what is where, then I know what is where because I am very careful about that. You'd really have to prove me wrong by ... well ... proving it. And not by telling that I don't know how things work or whatever (which I don't mean to say in a harsh way, the same as that you don't present this in a harsh way, ok ?).
So ... again you didn't go into anything of what I said, except that I judge wrongly. Let's twist this around explicitly know :

1. You know the theories, and you know them well.
2. The theories s*ck. Why ? because I say so.
3. I proved that, and you would too as long as you do what I presented.
4. ... which you did not.

Don't get angry because of my tone of voice or even "accusing" you here and there in this post. I mean well, but let's say I can't deal very well with your responses. Ok ?

Read better what I said (but hey, I write lousy, so I'm sorry about that !) and you wouldn't have written half of your last post for nothing (a waste of your precious time) :

If you mean that this availiable memory is smaller than the installed on the system is wrong cause you don't consider something else. Some hardware devices like motherboard, vga, network card, soundcards etc...

because I already said yesterday :

Lastly, there's also the mapped area for driver space, and this is the amount that is calculated from the total amount of physical memory minus the total amount of virtual memory. In my case this is 2048 - 1940 = 108MB.

and it is very clear that I here mean the exact same as you do.
Btw, don't confuse driver space with drivers causing issues in making the memory not available directly (mobo drivers).

Yesterday I asked you the a, b, c, d, questions because you keep on giving the feeling that the OS can use memory which just is not there. And, you just did that again. I mean, accoring to you (implied) first my 700 MB is virtual memory, and next it's in use by driver space. It's your choice, but please select only one of them. If you now think that my addtional 700MB (which is 800 MB including MY presented driver space) is driver space, then I wouldn't have any additional virtual memory, right ? (I hear you say : wrong ! )

the right is the opposite.
The less physical you have available the earlier stress time applies, because the translate to physical memory take place when it is going to finish.

PLEASE try to understand what I said about it. What I said about is is completely logic, explainable, and it also fits the practice for an explanation (short stalling of sound). Your response to that - and I am really sorry to say that to you - is again telling me how things work. Read-my-lips-please : I am telling how things workOUT. Don't avoid that !!!
Everybody who knows how to write M$ can tell that less memory causes complications. The last who wants to know is me. But don't you think it is rather sophisticated how I twisted that around ? Can't you see that it really doesn't help by saying that the truth is the opposite ? Of course, the planned way of working (hence the theory) is the opposite. But didn't I explain how theories flaw ?
So, when you don't agree, please tell me where I am wrong (that is, if you like to), and throw away the books now.

I don't know for me it is more important to understand how it works, and not to try to guess. As a matter of fact i 've try it and i don't get clicks at playback.

And what if I told you that you don't even know where to begin testing because you don't know at all what this is about ? What clicks ?
And whatever this is about (for you to find out), you might have missed that the person who is so ignorantly talking to you is the same person who doesn't have "your" clicks, whatever they are. Now what ?
-> Your quote above has become worthless. I don't get clicks either. who won the prize ? hehe

Furthermore (and related), this one I actually reject to be in these forums, because it creates confusion only :

With 4 gb memory and SWAP OFF i can play about 1,8 gb (170 min) without clicks. with 2 gb which means 1gb for system and 1 for apps, for sure you can play about 90 min of 44.1/16 without clicks in a system that have modified processes with the above tweaks. 90 min is not so little. When i say without clicks i mean for systems that don't have latency problems, because this is another story.
When we talk about 96/24 i think, that only if the program runs in native x64 can handle this situation and if we have enough physical memory 4gb or more.

This has nothing Nothing NOTHING to do with clicks or whatver it is you think, whatsoever. Not for XX. Please, if you want to bring up things like this for arguments to ... well, to what ? please be sure to have the right subject at hand.
I challenge you to find any post somewhere in here that talks about clicks or another playback anomaly, that you could dedicate to your statement above. But of course my moral is : you seem to respond here to something I said or claim, so it would be more easy for you to find that (should be somewhere in this topic, right ? ). So no offence meant (), but now pls not only try to keep out the theories, but also the means to prove them which are not related (AT ALL).

both of them are imposible to get full. If there is no much space then a proess just don't start!

Ah, it is very good that you mention this, because now we have something to talk about, to the sense of a possible big misunderstanding. I mean, many things "your theory" (ok, the books) claim, can be laid aside by stating the above wrong.
Can you tell me, where on earth did you got that one from ?

Of course this is not true ! :read:
Any program (and each individual process) can let grow the memory allocation step by step, and despite what I all said about it, I guess you just never looked at TaskManager what happens ?!? Or you did, but keep on keeping in your head that you can't know what's in there anyway, so you don't even start to wanting to understand ??

Oh boy ...

This is the last one, hoping really that you never use your books anymore :

Os have to take care of other processes as well. If for example a defragment starts, it needs both virtual memory and physical space that it has to remove it from the upper limit of 2 gb for user mode apps.

or IOW, be a bit more practical. Didn't you get how I am working on these matters from the How I tweaked my Vista virtually dead topic ? can you really not imagine that I take these things into account when I express myself ? Or IOW : that you tell me the obvious is one, but that you *think* all is to be placed in the context of simpleness ... please don't.
Btw, you are of course fully right with your observation.

Keep in mind the red sentence above !
I don't dare to read back, but I'm afraid you can only feel offended. Please don't, I don't intend that. It's my only way of dealing with your good intentions, and I'm dutch of course.
And don't forget, since I read into it, we just agree (better : I agree with you, because you were first). The workOUT is different though, and a 1000 times more complex than the theory of the elements.

Oh, since yesterday I switched of Paging. No anomalies so far.

[astacus21]

Don't get angry because of my tone of voice or even "accusing" you here and there in this post. I mean well, but let's say I can't deal very well with your responses. Ok ?

No Peter for sure i don't get angry. It' a forum everyone has his opinion and it is respectable by me.
I enjoy this convertation as well, and the only reason that i get involved, is because i like your program and i want to help, to become even better.

Many times, you talk about situations that happens to your system, but you don't understand that i can't simulate them and see the same things as you do. For this reason i avoid to talk about them and i prefer to talk about things, that are system independable. So you make the mistake to believe, that i don't like to get into this or i shrug off your arguments. The only reason that i didn't answer all this is because i can't simulate the same circumstances as you do on your system.

Another thing, is that you blame me that i repeat a theory (sometimes you say my theory!) that after the experaments you make, you verify it.
The only answer to this, is that it is not my theory, but the way that OS acts. You may known many things about it, but when i write something in a forum i'm not indebted to know the knowledge of everyone. So i tried to explain from the begining, how the memory management acts. For sure this was the first object to discuss and then, if we agree, we can go to the next step and discuss about details.

After so many posts i'm not sure if we agree to the basics. And this is a major problem, because if we don't agree, it has no meaning to talk about small details.
For this reason i say the same things and you think that i don't get into your questions. For example you ask me about a process of 5gb, when i've said so many times that you can't go over 2gb. I don't answer to this because it is pointless. It has no meaning to b answered.

For the other things you say above i don't want to answer, because i feel that i've allready done it. I understand that you may doubt for the answers but ok i can't do much about it. I have said my opinion and i don't feel bad if someone has a different position.

Also when i said that i don't have click and pops, i didn't mean that you have. But for me this is a proof, that now my system works better because with swap enabled i got some clicks. So please don't take it like this. This was an observation that has to do with my system.

You say also for this ''both of them (vm and pm) are imposible to get full. If there is no much space then a process just don't start!"
(remember that we were talking about a 4gb system)

that

""...Can you tell me, where on earth did you got that one from ?Of course this is not true...).

You mean that you can start a process that exceed the 2gb limit?? or something else that i don't understand?

Well there is no reason for me to defend or the opposite. If this convertation doesn't help to understand something then we should stop this.

I was also ready to propose another tweak for vista, but if we have to talk about it for days then i should better keep it for myself 
(i hope to take this as a joke)

[PeterSt]

Nah, I guess the language barrier is bigger then we (or I) thought. It doesn't seem to be a matter of not wanting to answer, but merely about some 5GB you mention now, while I don't even recognize what it is about at first glance. That is, assuming that you are talking about the a,b,c,d, questions, I took 4GB for a limit because you have a 4GB system, and didn't think of that other 2GB limit. For that matter (I realize now) I couldn't even turn my a,b,c,d, into 2GB (and 2,5GB) because it would be the same problem. Also they can't be turned into e.g. 1.5GB (and 1.7GB) because that wouldn't make sense to the whole story. What story ? pfffff
Edit : Btw, language barrier, or not paying carefully attention from my side ? hehe

Also when i said that i don't have click and pops, i didn't mean that you have.

That is another example. This is not about me having them or not, it is about the subject itself which is not appropriate unless in the exact right context you don't want to discuss (well, I do it, and you don't respond to it). So this is talking miles along eachother.
And for the contents of this real subject : this can NEVER be about one track (testing). You need more, maybe 10. Somehow this goes right along you, and that is why you are not into the real subject from the beginning. And, this is why we never talk about the same. If this was about one track, I (and NOBODY) wouldn't care less how the swapfile behaves, or what happens without the swap file. Then it could only be about the size of one track (like you are testing). Big deal !

You mean that you can start a process that exceed the 2gb limit?? or something else that i don't understand?

So indeed this is another example, and I can't understand what was unclear about that. But keep in mind please : I don't say that it's your fault; it's way bad communication which I now call the language barrier. But if you tiny winy allow me to be the psychologist : it can be well be so that this (underlaying subject) is so fixed in your mind that you just cannot read what I say : If a program is too large it won't start you say. And I say that a program can begin small, and later grows. Is this now clear ? So when it starts small it sure will be allowed to start in a memory area which lateron becomes too small because the program needs more. This is so basic ...
Besides that, XX is constantly acting like that. I told it, you can see it for yourself, it is all over in various topics ... and it is the subject of the problem. What problem ?


We're kind of helpless I'm afraid. Hahaha


PS: If you have clicks with the swapfile on, your system isn't perfect. If this is during playback in the middle of tracks, your PC is doing things it shouldn't. Solving that by shutting down the swapfile is the WRONG SOLUTION. First something else is using the swapfile and/or that works at the wrong priority. And yes, I too use the PC during playback, hehe.
So, that was for distraction. 

and i want to help, to become even better.

I know, and it is more appreciated than you might think.
Also, you forced me to dive into things I didn't do before.
Now if I'd only solved your foolbar problem ...

[astacus21]

You are right about the language barrier (as you call it). I think that even if we were british, we would still have a communication problem through a forum. Sometimes you say something with confidence that it is true and you get an answer that makes you understand that the opposite side have understand something completelly different.

About clicks and pops, i have explained somewhere above, how you can have a performance drop that could cause clicks at playback. And i thing that i have explain it very well. This performance drop (or undesirable cpu usage) is independent, from how many track you have load, or if you are using xxhe. or flight simulator. Maybe you have something else in mind when you tell that something happens after 10 tracks or so, but this is unrelated to this performance drop i described (when, how and why it is happening) and it is easy to observe it at task manager. So if you had something else to your mind why you didn't just tell me about it? You never know i might have an answer relative to your subject or maybe not.

When a process of over 2gb don't even start, it is self-evident that if a process grow step by step to over 2gb it will close with an error. I thing this is obvious. But i admit that i haven't understand that you mean it that way.

About your PS i just thing you are wrong (how many times i said that?)  .

PS1: I 've read again all the messages and i think that i have understand why we have so many misunderstandings. I'm talking about a 4gb system and you talk for a 2 gb system. In your case if you disable swap you have 1 gb space for kernel and 1 gb for apps. this 1 gb for sure can get full and make a process stop. Now please try to understand why in a 4 gb system, this space cannot get full by a >2gb process.
the first thing i told you to consider is that we have a 4 gb system cause this situation contains all the others.

PS2: I want to inform you that i can listen to xxhe with q1=-4 while at the same time i watch tv with dscaler and i measure latencies around 45 μs with peaks at 65μs from the begining of the track to the end. Not so bad for my non perfect system.

PS3:

Now if I'd only solved your foolbar problem ...

except the 24 bit problem with xxhe, i have also another one with volume control but i'm afraid to ask 

[PeterSt]

I keep on going, as long as you can bear it.

About clicks and pops, i have explained somewhere above, how you can have a performance drop that could cause clicks at playback. And i thing that i have explain it very well. This performance drop (or undesirable cpu usage) is independent, from how many track you have load, or if you are using xxhe. or flight simulator.

Difference between theory and practice. I, in my system, I often have three or four bulk (SATAII) copies running (no disks are shared), and I can start a loop or two that eats officially 100% CPU, and there's not any anomaly at audio playback. Ok, I don't say that it is the best for SQ, but no anomalies (no clicks, pops, glitches, nothing). This immediately makes your explanations without value. They may matter to you, but not to me and because they do not matter to me, they should not matter to anyone once their system is, well, right (whatever that exactly is). Also, whether it is defender, prefetching b*ll, Indexer, and filling the swap file all together, my playback stays stable. This is not only on Vista with Engine#3, but it is also with Engine#1 and #2 on XP (or Vista).
I trust it you believe me, and it is not important that this is so, or that I am "good" or whatever valued can be connected to this. It is about one thing only : it can be done, and a stupid spike in the cpu does (proven !) NOT matter. What does matter, however, is the priority of that spike, and this is largely out of our direct control. Indirect it is though, and the start of the description of that is in my first post from today. Whether you can agree with that or not is up to you (although I'd like to really make it clear), and fact is that *I* will use it for myself (read : XX).

Maybe you have something else in mind when you tell that something happens after 10 tracks or so, but this is unrelated to this performance drop i described

Yes, of course I have, because that was in that other topic (Edward), and with my description just above you see that we indeed make our own subjects. However, you started the subject of the swapfile, I said it is not necessary and (hence) not advised, while it is you bringing up the other subjects (e.g. your clicks) and it is may hijacking your subject (no swap file) to testify that that just can create the clicks (at track boundaries, read the other topic). So again :

Maybe you have something else in mind

I try to stay at the original topic ...

When a process of over 2gb don't even start, it is self-evident that if a process grow step by step to over 2gb it will close with an error.

I knew you would agree on that. Sadly, XX is not growing setp by step to over 2GB. THE OS DOES.  (read my posts again if this sounds unfamiliar to you, and mark the over 2GB. So yes, XX grows, but not over 2GB. The OS does that for me, and it does that in the VM area).

About your PS i just thing you are wrong (how many times i said that?) 

It is not important how many times you said it, because I will never ever agree. Why ? because the principle of needing to shut down the swap file is wrong (and *that* is because we can't add memory for ever and ever). Besides that, and this is the most important, I don't need to shut down the swap file in order to avoid the clicks - which are not clicks but small silent gaps (did you already read that topic ?). If you need to shut down your swap file in order to avoid clicks you are the only one needing that. Did you hear this ?
Do a search, ask the question, I don't mind. But *might* you find posts about clicks, be very very sure this is your situation.

this 1 gb for sure can get full and make a process stop. Now please try to understand why in a 4 gb system, this space cannot get full by a >2gb process.

Can you imagine I don't care ? Oh, I care about that fact allright, and I care about that I learned this (from you), but it is irrelevant to your first post and my first response.
Besides, you rigidly do you approach this actually ? I mean, you say that a process that dies on a >2GB memory need doesn't suffer from an error of crossing the 4GB limit. I know, I twisted your words, but I sure do say the same. Or better, this is what you say. But please don't, because nobody can understand, and the last one is me.

PS2: I want to inform you that i can listen to xxhe with q1=-4 while at the same time i watch tv with dscaler and i measure latencies around 45 μs with peaks at 65μs from the begining of the track to the end. Not so bad for my non perfect system.

Yeah yeah yeah, you are joking. But do you want to know the truth ? Here it is :
Re: Check to see if your computer can cause drop-outs
The first graph is the one of my super system, standard 1000us.
So don't get inconfident, but please learn about theories not being right, or at least being incomplete.
Of course I am making fun here, but the truth is that this is my system, and thus also the truth is that *or* this picture (hence that latency checker) doesn't tell a thing *or* that 1000us is suffcient to even add the copying and loops I described above and still don't notice that.

Now, I beg you to understand what I call stress time, and which you are not allowed to map onto your thinking of what stress times should be. Keep in mind that all is prioritized, XX does all what it can to do that the best, but that some OS processes interfere with that, which processes in the end *have* to have the highest priority or otherwise things go wrong. Your presentation is way too simple and you only say a cpu spike causes this. Now, ban this from your mind, or bet a million with me and come over (I'll pay the ticked and you lose).

It is my humble suggestion that we (at least I) learned enough of this,and respect eachoters thoughts and goings. I sure respect yours, and I already am sure the bet will be off because long before the millions are grabbed together there will be some "aaah" and "oohh" and "did you mean *that* !". I recognized a few 1000 words ago that we agree, and said that. I am still confident we do, and I mean what I say.

except the 24 bit problem with xxhe, i have also another one with volume control but i'm afraid to ask

That looks very wrong, and obviously is my fault. But please open a topic for it, and I'd say it can't be that hard when all is fresh and new, and I pay very good attention to what you have to say.

[PeterSt]

I just looked, and XXEngine3 indeed is compiled for X86 too. What I recall though, is that I did this for safety and it never was proven that it is necessary.
Besides that, coincidentally today I most probably found why XXHighEnd wasn't compatible with X64 in the first place. But that's for another time tro try.

I just tried to build an explicit X64 for you (in case you want to try), but there's an external dependency on X86 somewhere.
Later ...

[PeterSt]

But ... (I just can't stop ) here's a thought for you :

Tonight I played an album which consisted of one track, just larger than the space of additional virtual memory. What happened  ?
Allocated physical memory (the green figure) and virtual memory (the "swap file" figure) were equal. Both 1200MB. I am fairly sure that if that track would have been 200MB larger (beyond normal available physical memory) I would have receive an error.
Fairly sure is not 100% sure.

Also, and I cannot prove this yet to be related to my "issue", these kind of tracks almost always end in an error (XXEngine3 stopped working) of which I can't find the reason. This happens when playback has finished, hence (???) the memory is freed.
Highly speculative, because I myself can do things wrong. Nevertheless this is the case, say, 90% of occasions when such tracks are played.

[PeterSt]

I am fairly sure that if that track would have been 200MB larger (beyond normal available physical memory) I would have receive an error.

Which I could incur for by playing another such track right after it ...

This is all about precise math (the 2nd track is 318 MB which would be needed twice temporarily), but I dare to take the conclusion that via-via-via this track should have fitted in my 1350MB normal available physical memory, which was 1200MB full when that (2x) 318MB track was loaded. The additional virtual memory seems to have shut off completely, once it reached its own limit, WHICH WOULD BE THE MOST LOGICAL TO ME BECAUSE IT WOULD HAVE BECOME INCONSISTENT ONCE IT HAS TO THROW OUT THINGS FROM THERE.

Something to sleep on.

[PeterSt]

However, before I fall into my bed ... it would be more fair to say that the one process can't stuff the total amount of 1200MB in the virtual memory, so the OS didn't even try. And remember, the OS wouldn't know about parts of that process to swap out, so its all or nothing.

Conclusion (whatever it's worth) : when it's not the OS that lets the memory grow (remember, that happens track per track when they are small) but it's the program itself (this exceeds my 1350 available physical memory now), I get the memory overflow.

Now, do you want me to switch on the Swap file in order to test this and see whether then no error occurs ?

[astacus21]

Of course i believe what you say and i understand from the graphs that you have fine tune your system, so that it has very good performance in the pci bus. Once again i say that this cpu spikes MAY cause clicks, but i understand that you can have 100% cpu utilization without clicks and pops, if the process that causes this spikes, runs in lower priority and not for very long time. I' ve also see the same behavior- results in my tests. But i think you agree, that 100% cpu utilization is not the best thing to happen. So i propose this tweak because with the swap file disabled, you decrease the possibility for this to happen.

Something also important relative to this, is that at vista (not at server 2008) the biggest priority (as demical value) is 31. And this is the priority that have the critical kernel processes. Keep in mind that the realtime processes have the value 23-26, so every critical system priority can produce clicks and pops when it is active for long time. I hope you will find this helpful.

I try to stay at the original topic ...

ok but you didn't say that it was something else. Sorry to bother you with this.

Sadly, XX is not growing step by step to over 2GB. THE OS DOES...and it does that in the VM area.

I'm not very sure that i understand well this one. You say that you expect lower memory utilization for the process and you see unexpectidly bigger?. It is not so important what you see, as a size in virtual memory. When this come to the physical memory has the same unexpected size?

It is not important how many times you said it, because I will never ever agree.

It was a joke. But i'm sure that we will never agree to this one hehe. Don't worry about my swap, i dont have clicks, i get pure sound.
So now i know what you mean with stress time. The mystery parts of our convertation find their solutions. So you want at the end of the xx process to change the priorities of those to be more than xxengine. sounds difficult.

Yeah yeah yeah, you are joking

No i'm serious i get this numbers with vga drivers installed. if i uninstall them i get under 20μs but i can't create overlay for tv. You believe me or you want picture for this?  

please open a topic for it, and I'd say it can't be that hard when all is fresh and new, and I pay very good attention to what you have to say.

OK i will thanks. I just want to try with some other soundcards first. I think that it is a driver issue.

Now, do you want me to switch on the Swap file in order to test this and see whether then no error occurs ?

Your limit should be around 1gb with swap off. If you enable it, your limit goes to 2gb, but with some performance drops when the system releases physical memory and gets new data from vm.

I just tried to build an explicit X64 for you (in case you want to try)

of course i want, but take your time. there is no hurry.

these kind of tracks almost always end in an error (XXEngine3 stopped working)

I get the same warning when i can't start a bigger than 2gb wav. So it must be a running out of memory issue.

the additional virtual memory seems to have shut off completely

the difference of the virtual memory is tha the data are flat (continious) So it is logical to become corrupted when system clears some space.

[PeterSt]

Of course i believe what you say and i understand from the graphs that you have fine tune your system, so that it has very good performance in the pci bus.

Now I don't understand. Which graph ? I referred to a graph that shows 1000us for standard, and now you call this good ? It's the most lousy ever, and all still runs ok. If it is your conclusion that I thus must have fine tuned other things etc.  ... no.

but i understand that you can have 100% cpu utilization without clicks and pops, if the process that causes this spikes, runs in lower priority and not for very long time.

Same priority is OK at continueous 100% useage. No, it's not the best.
If sound gets stalled (hickup) this is not about CPU useage but about interrupts and how the CPU has to deal with *that*.

so every critical system priority can produce clicks and pops when it is active for long time

Please don't make up things you didn't encounter yourself. We are talking XXHighEnd here, and you did not hear clicks or pops or anything, because at this moment you cannot describe them to me. So, get yourself familiar with what is a click, and what a pop, and what a glitch and what a hickup and what a stall and what a continues repeat and what a tick and what crackle and what cracks. They are all successively described and act as they sound (hehe) similar to video terms, and you use the terms randomly with causes which aren't true. In your books yes, but I asked you to throw them away.

Sadly, XX is not growing step by step to over 2GB. THE OS DOES...and it does that in the VM area.

I'm not very sure that i understand well this one.

I explained the sequence twice. Start with looking for something like "every byte that goes into normal available physical memory goes into additional virtual memory as well", which was the second time.

So you want at the end of the xx process to change the priorities of those to be more than xxengine.

You didn't read yourself into that topic I pointed at a 100 times. If so, you wouldn't have asked this silly question.
IOW, that's already in there.

No i'm serious i get this numbers with vga drivers installed. if i uninstall them i get under 20μs but i can't create overlay for tv. You believe me or you want picture for this?

I sure believe you, and I just tried to make some point there. Look at my first remark in this post ... somehow we both switch things upside down. Never mind, as long as you understand that my system is the most lousy (and this is no yoke).

the difference of the virtual memory is tha the data are flat (continious) So it is logical to become corrupted when system clears some space.

Beware what you're saying, because one step further and you're going to admit that it all can't work.


What shall I do, close this topic, or let us keep on going with word games ? 

[astacus21]

Nothng to reply here so this must be the close of a super misunderstanding thread.

Just some observation...

For the graph i thought that the 3rd one was after the tweaks.

I see you have many words to describe a problem in playback. I used 2, you use (let's count) 9!! maybe there others too!

OK after this last post i'm going to burn the book, so that it will never cause a problem again. I think the difficault part is to find where it is...

Also i want to say that it is no matter of lousy or fine system. If something is working fine there is no need to be state of the art.

And finally about the vm, once again i 'll say that it has a sophisticated way to deal with many (normal) processes. It is not there for use by 1 proccess. This is something that was happening in the past. (Dos ages)
If you have the swap enabled, this data will never get corrupted, because there will be all the time, space for the max (2gb) an app can use.
For this reason it does't have a limit of space, you can use whatever you want 5gb or 10 gb in your hard disk. If you have 2gb of physical (and swap off), yes it can get corrupted, the same if you have 4gb and many many apps running in the background.

No need to close the topic. Who knows maybe one day, we may find useful things in it.

[PeterSt]

All agreed.

And no, that third graph is my (indeed PCI tweaked) XP HTPC machine. And FYI, that serves so many processes that I forgot them. For fun I name a few : IIS web server, heat control (via OPC might it say something to you) with hundreds of sensors, it runs satellite receipt (MyTheatre), DVD etc. movie playback (TheaterTek), servers Photoshop and stuff, all the Wave analysis tools, digital recording, DVD/Audio ripping tools, photo storage (over 100K photo's), serves network media playback (networked player, networked TV), audio calibration (think of subwoofers), anime creation tools (like icons, smileys), is the printserver, and some other stuff which count some 150 icons on the desktop.
This is not my audio PC, and it looks like I'm the only one on the planet who never saw a stutter (video) coming from it.
Also, this is not some kind of show-off, but it proves what can be done from a miserable P4 with hyperthreading shut off and 1GB of memory, as long as you carefully watch what happens when, and what causes which. The key is really there, and instead of reading the books or copying statements of others, I write the books. Hahaha.

For some trust or good feeling on your side, I was the very first on this earth that started and introduced legacy Enterprise Resource Planning on PC's , when the PC's were 12MHz and the servers (it needed two) were 33MHz, in 1987. That configuration served some 100 concurrent users, performing over 100K transactions per day.
A show off afterall ? ah, maybe, but at the time Microsoft bought Fox Software (the tool I used) and turned it into FoxPro, which later became Visual FoxPro, and which contained 1000nds of bugs I (my company) had to work around but sometimes couldn't, Microsoft told me that the tool was meant of 10 users max. ... but I had customers with 250 users and I gueranteed that the 250th user wouldn't notice the difference opposed to when he was alone in the system. Remember, it is the MS books you are reading !
Today you find this in e.g. the Library functions which is based upon the exact same (API stuff) as Explorer is based upon. You may need over 10000 albums with pictures (which I have here) to notice, but what happens in the Library area for speed is undoable in Vista itself including when the files would be indexed by Vista. It is not difficult to outperform the creator (MS) with his own tools, as long as you recognize the flaws, watch carefully where they are, and then avoid them (creativity comes first here).

Now you know.

Mr. ShowOff.