Ah, HiFace - so you're in USB camp....
Are you saying 44.1 does not work in W7? That would be interesting....

In meantime, couple ideas you probably tried but just to make sure - Have you:
- manually turned off Power Management for every USB Hub in your system? (see picture - by default it's on in W7)
- made sure you don't use any software that dynamically manages your CPU speed (e.g. RightMark) i.e. Power Plan = High Performance?
- tried disabling SpeedStep in BIOS?
- tried 'USB polling' tweak from couple posts above? (I'm a bit skeptical on that one but you never know....)

It does seem so, yes.

In principle, W7 _will_ resample as needed if you are not in KS or WASAPI exclusive mode but XX or even Foobar are both using those modes. However, if you select DirectSound in e.g. Foobar, all bets are off and chances are your bits might get resampled depending on your sound card. (BTW it would still sound drastically bettter than XP as resampling is done in 32-bit float as opposed to int16 in XP)

The difference in SQ is coming from somewhere else - I have tried compiling official Microsoft sample source code for WASAPI exclusive mode and while they report same latencies in both W7 & Vista the music is completely unlistenable in W7 due to glitches & skips! (not to mention SQ).

Why is there such a difference between W7 & Vista I'm afraid nobody here could explain....

Maybe it is related to drastic changes in W7 scheduling algorithm (AFAIK MS completely rewrote that part), maybe not... But you can try one thing which may make W7 more bearable: Try changing 'processor scheduling' to 'Background' as opposed to default 'Programs' in Control Panel - System - Advanced system setting - tab Advanced. Let us know if you perceive any difference!

You'd need original Vista disk for vLite, yes (if you have a license key you may be able to download it from MS if, for example, you have an MSDN membership).

But if you already have Win2008, stick with it - as mentioned, binaries are identical to Vista SP1 so you can safely use it for comparison with Win7.

At the risk of sounding pedantic: You can never 'disable Virtual Memory' - it's not possible. You can only disable page file creation.
And this setting does not disable page file: rather, it disables paging of OS Kernel which is always safe to do as it is rather small (btw probably would have been in OS cache all the time anyway, but let's make sure).

Out of curiosity: What is the reason new version will recommend having a page file? (as: no page file = less I/O behind the scenes).
Is it because you are implementing that suggestion with having XXEngine constantly resident in RAM with potentially large cache and would like to avoid situations where users might run out of (virtual) memory? So I'd assume those of us who don't plan on using computer for anything else but music playback can leave it off, or...?

Me too but I can't find time to make a fresh install... Post your findings when you do!


http://www.vlite.net/download.html


It's very simple and will guide you step by step. If you want to strip Vista to the bone so it's only good for music just do the following:

1. Specify folder where your original Vista ISO is and a folder where 'light' Vista will be put. vLite will copy all 2.46GB in there so make sure you have plenty of free disk space!
2. Tabs on left side will be activated (Tasks, Integration, Components etc).
3. Click Components: here you can select which components to remove permanently i.e. they will not even be on your new installation disk much less installed!
4. Select all these: Accessories, Drivers, Hardware Support, Languages, Multimedia
5. Open Network and select everything except "Link layer topology discovery" & "Quality of Service"
6. Open Services and select everything except "Desktop Window Manager" (XX will crash without this) "Diagnostics" (not strictly needed but allows you to see how 'quiet' is your system) "Multimedia Class Scheduler" (not needed for XX KS mode but if you want to also try e.g. Foobar then leave it - it's perfectly OK) "SSDP Discovery" "Universal Plug & Play" "User mode driver framework" (just in case if you have an older driver)
7. Open System and select everything except "Microsoft HTML Engine" "Performance Counters" "Reliability & Performance Monitor" (again, like 'Diagnostics" you don't strictly need these but they can help pinpoint issues)
8. Click Tweaks and set Security->DEP always off, System->Paging disabled, hibernation off, power scheme->high performance. Optionally set desired services to 'manual' but you can just leave it as is.
9. Click Unattended and fill in your key etc so you won't have to do it during install. Make sure you click 'Skip user creation' and leave Administrator password empty (it will boot faster as you won't have to type password)
10. Click Apply and you're done!
11. After 'light' Vista is created (it will take a while, go get a drink...) you can either copy resulting files to USB stick (note it has to be bootable otherwise it won't work!) or create an ISO and burn it to DVD (well, it might even fit on CD (!) Think mine is only 728MB - compare that with 2.45GB at start!)

Gotchas:
- If you have Vista 64bit you must run vLite on 64 bit OS.
- if you get errors when starting vLite you might need to copy wimgapi.dll to vLite folder, see here:
http://webcache.googleusercontent.com/search?q=cache:0fh-gJJnnp4J:setiathome.berkeley.edu/forum_thread.php%3Fid%3D50222+vlite+missing+dll&cd=2&hl=en&ct=clnk&gl=nl
- I am not sure if vLite will work with Vista SP2 (have only used it with SP1)

After these steps you should have 0 stability issues with Vista and, in fact, you'll be hard pressed to find a difference compared to 2008.

Chris - 'more stable' etc....this is a myth.

Server 2008 is identical to Vista SP1, just configured differently (as you correctly suspected: 'it's stripped down').
In fact, MS released Server 2008 at same day they released Vista SP1 and they even called it Server 2008 SP1 (!) which confused a lot of people - See here: http://blogs.msdn.com/b/iainmcdonald/archive/2008/02/15/windows-server-2008-is-called-sp1-adventures-in-doing-things-right.aspx

Don't waste your money on 2008 - get much cheaper Vista, use vLite to strip the cr*p out and you are 99.99% there - spend the saved cash on some nice music & wine ......  (btw the remaining 0.01% you won't need or can get by couple registry tweaks).

 

Cores are oblivious to both slots & interrupts.

AFAIK every PCI slot has multiple interrupts assigned (think 4) and each PCI card can use as many as it needs. But those interrupts are not what you see in OS. Hardware PCI IRQs are mapped via IRQ router on your motherboard into IRQs you see in OS.

In theory, you may be able to fix PCI IRQs mappings from your BIOS but that is not guaranteed to work as Windows is Plug-n-Play OS and can shift IRQs around as it sees fit. You may try disabling PnP of course, but that could lead to all kinds of other problems and in any case there is no need for any of this.

I may be wrong on some of the above (not my field) but it really is not that important for our needs as there is not much we can do about that process except to keep in mind that different PCI slots by design get mapped to different IRQs (as you just found out) so if you move stuff inside your machine you may need to re-visit Interrupt Affinity tool.

What is important from our perspective is where IRQs are handled in the end as, by they nature, they will _interrupt_ anything that is going on at the moment. And we probably don't want music player process being interrupted just when it is busy filling sound card's buffer especially if we are using ultra low latency do we?...
What follows is that higher latencies (=Adaptive mode) may not be as much affected as Special mode (may not even be perceptible although I believe it is) but as a matter of principle it makes sense to move everything possible out of time-sensitive process i.e. particular CPU core.

Since Windows by default will not give any guarantee which CPU will handle particular IRQs (as it tries to balance it out evenly between cores theoretically it could even switch it _during_ playback) that is the reason why we need Interrupt Affinity tool: to bring order to chaos and make sure time-sensitive things are not needlessly interrupted all the time.
 

 

Only if I get an invitation to DAC audition and get a glass of nice red wine too!


Believe it or not NO but I see now what you mean and I'm afraid this time you misunderstood me (for a change).

I can see you want ultra speed to write to memory so this CPU intensive 'stressful' task of converting a track (well only if you use 8x Arc etc otherwise it's no so stressful but OK lets assume worst case) can be done as quickly as possible - but you don't, repeat, don't need same speed for _reading_ the file from disk - in fact, you cannot have it even if you wanted it. What?
Well, as we seem to misunderstand each other frequently let me try to explain using an example:

- Suppose we have XX written as a device driver (_not_ a 'RAMDisk' - let's call it 'Xtra Xeroxed audio' driver or 'XX driver' for short ).
- XX driver has a defined RAM size that it uses as a music data buffer - just like you would set up your RAMDisk size now
- Unlike RAMDisk however, XX driver does NOT appear as a disk D: E: F: whatever, because it is NOT an implementation of disk I/O interface (it's an 'audio playback' driver after all - just repeating to make sure I'm obvious)
- access to XX driver is controlled and only possible via XX GUI. Otherwise it is completely invisible (except that your usable RAM is visibly reduced by XX driver's buffer size also just like with RAMDisk now)

Ok, that was easy - But how does it work?
In 6 steps:

Step 1: GUI starts working just like it does now: prepares *.dat/*.dao instruction files and sends them to XX driver (You'd have to use a proprietary API or whatever method you like (oh yes, MMFs could work too, LOL) as you won't be able to 'just copy a file' because XX driver does not have a file I/O interface at all!)

Step 1 addendum: BUT, when 'Copy to XX' is checked it does NOTHING! (apart from noting the setting with all others in dat/dao instruction files it just sent to XX driver)

Step 1 addendum 2: XX driver (which replaces what is now XXEngine3.exe) has full, absolute, unlimited control over its allocated buffer! It does NOT have to go thru OS file I/O stack to write to its RAM! Hence, it can do ultra-fast conversion in its own buffer which is only limited by speed of the computer. IMHO This is not really that important (see Step 3) but I'm making a note as you seem to think it is.

Step 2: So, our perky XX driver opens tracks in playlist (specified by dat/dao) via MMFs! (I can hear you: 'Oh my God here he goes again I just told him it's baaaaad, noooooooo......')

Ahem, why MMFs? Because that way XX driver needs 0, repeat, zero memory to read _all_ tracks regardless of their size! So we can have XX driver set to use 3GB RAM if we wanted to, and we do, so damn the RAM we want more music! (notice, we do NOT have nor will ever need a RAMDisk!). And if we had 8GB RAM we'd make a bloody 7GB XX driver because we're audio freaks and have dedicated audio computer damn it etc etc. you get a picture....(not that it wouldn't work on a 2GB RAM Intel Atom notebook but our XX driver buffer would naturally have to be proportionally smaller....)
 
Step 3: As XX reads in tracks one by one (for sake of argument, let's say it reads 10MB 'chunks' at a time - it really could be anything, does not matter as you are limited by HDD/SDD disk speed anyhow) each 'chunk' gets 'converted' right there 'in-flight' and bytes fresh & ready for sound card are put in XX driver's internal buffer starting from offset 0.

Step 4. Playback thread starts playing music as soon as first chunk is 'converted' OR as soon as buffer is completely full (similar to current 'start engine while converting' setting so it's under user control)
In any case, loading thread stops loading/converting when buffer is full.

Step 5. Music plays, and plays, and plays and generally absolutely nothing interesting happens....track 1, gapless into track 2, I did it myyyy waaayyy, track 3, she loves you, yeah, yeah, yeah, track 4....track 7?8?9?

Step 6. Ooops, XX driver sees there's only, say, 20 seconds of music left in the buffer - what do we do? Panic! Or maybe not: Go to Step 3, repeat until playlist exhausted.

When playlist is done (or if user stops playback) XX GUI gets reactivated same as now.

Notice when you start new playlist absolutely _nothing_ has changed in memory layout: XX driver is there all the time with its ultra fast ultra simple (single!) buffer and will just write new playlist content over previous....no allocations, deallocations, garbage collections, VM swapping, no nothing.....just music playing forever, undisturbed............






 

I already sent previous post without seeing your latest:

I'm sad to hear you saying you're not open to ‘well meaning advices’ but it's your forum and you make the rules here so I guess I'll have to shut up.

I hope it's fair though to let me respond to your latest post?

Fair enough, it's your product.
Considering the ease with which this can be tested I'd personally at least try it: even if it does not work in terms of SQ it might reveal something interesting that was not known previously which just might be useful.... But then, that's just me being curious. I guess I'd have to do it myself in order to find out, LOL...


That is the point I was hoping you would catch on and IMHO a single most important improvement to XX (or, rather, to audio players in general) with potentially greatest benefits:
Look at your RAMDisk. It is nothing but software. A device driver to be exact. And it sits right alongside OS. In fact, it is on 'per tu' basis with OS Kernel.
Compare that with current situation of XX being a 'user' process (as opposed to system) running in what is essentially a virtual machine on top of OS (.NET).
What if XX could be written as a device driver (a la RAMDisk) but with a twist: It 'knows' about its contents not being 'virtual disk files' but sound data that can be directly accessed without having to go through OS file I/O layer at all?

And by virtue of, effectively being a part of OS, what other layers have also been removed? (not having to deal with VM much less garbage collection etc etc.). And because it sits there (like RAMDdisk) it is 'stable' and forever unchanging (except for contents, of course )

Now, how much closer is that to ‘bare metal’, what new possibilities would that open and how far more could XX go and sound then?
Puzzles the mind....


I see that you have caught on this one too: I was hoping you'd see the possibility of using MMF to 'replace' *.dat & *.dao files.
I don't, however, agree that it is 'ridiculous' at least as a matter of principle:
I believe most people here can agree that introducing RAMDisk brought positive changes in SQ. And, as you said yourself, it is not clear to you (not to anyone else, me included) just WHY is this so.
Is it a crazy notion to suppose that it has at least something to do with eliminating SDD/HDD disk-induced I/O interrupts as such do not exist with RAMDisk?
And if this at least sounds plausible, wouldn't using MMF instead of writing/reading *.dat/*.dao to disk be beneficial purely on principle as it eliminates disk I/O? (Yes, I know it may be a moot point if dat/dao are being written to RAMDisk but it still _has to_ make a difference as path through OS is very different. Maybe that difference turns out to be small so it's not worth it but would that make it 'ridiculous'?)

BTW - I may be wrong (I don't have logs with me) but if I remember correctly when you do either write or read of dat/dao files they had 'non-cacheable' flags set. Those files are so small they can fit quite nicely in OS cache and, as free bonus, you are likely to avoid file I/O interrupts when reading them back in from the Engine as there is high probability they'll be served from cache. I.e. you may get MMF benefits (at least in this case) without having to use MMF (since you seem to have a particular dislike for them.)


I don't know! And you don’t either! That's the whole point, LOL!
What I was (obviously unsuccessfully) trying to communicate is that if there is an interesting approach that has not been tried before it might be worthwhile to check it out (if it does not require too much time). Obviously, a proposal mentioned above, to write XX as music-player-device-driver is a LOT of work but trying out MMF really isn't. (BTW If you interested in pursuing device-driver approach maybe some people would be willing to help just because they find it interesting – for example maybe it could even be me (God forbid) ....)

But to try to answer your question in a more concrete way: I speculate you would _not_ have to worry about speed. I have read about projects using MMF to provide type-ahead suggestions (you know, like when you start typing on Google and it provides suggestions in drop-down as you type) on very, very large files: order of magnitude larger than any sound file you will EVER have - bigger than 32 bits 384kHz Beethoven's 9th gapless LOL And they had to do dynamic binary search on that file too, so they were jumping back & forth around that mega-giga-tera file without problems, where, in contrast, you only have to move sequentially forward.
And OS is prepared for it, so it will preload SFS sections before you need them - OS is sometimes not that bad you know and they have removed most of code Bill Gates wrote in his days....    


Yes, it _has_ to be slower as you are going through whole OS file I/O stack which can easily be seen in debugger: there is _no_ difference compared to SDD/HDD disk access so it can never do 500MB in 0.1 sec but why would you need that?
Sound card doesn’t need nowhere near that amount of data in that time period even with 8x oversampling etc. And you can ask MMF to open a view on whole file instead of SFS chunks - As mentioned, it would read in as you move forward and you would not need to manage memory because MMF memory _does not count_ in your process set! So we could have even bigger RAMDisks as XX would be using only couple MBs

But OK, I guess I misunderstood the purpose of this forum: I foolishly believed it was a place to discuss possible improvements to XX in particular and explore new boundaries in computer-based audio reproduction in general. Seems I got it all wrong so I apologize and promise to shut up.

And, since I'm probably _really_ getting on your nerves by now (hope you can see it's with seriously good intentions although my tone sometimes may not show it )  may I suggest that you do _not_ kill XXEngine process every time playlist is done?

Why? Well, now that you seem to have solved your .NET memory management issues (I'm guessing by sticking to blittable types and pinning down SFS buffers?) and seeing (hearing) tremendous SQ improvements (as you reported) I am sure you will be able to see the value of not having to 'recreate the universe' every time a new playlist is started:
By having XXEngine 'stick' around with initially allocated RAM you will be helping OS minimize memory starvation issues by simple fact of not working against it! (which I'm afraid you are doing at the moment by killing & restarting Engine with every playlist which inevitably increases memory fragmentation....)

 

Nope: no upsampling, no funny business: just a single 'straight' 16/44 WAV track.


So you're saying that's where those 3x100MB buffers come from i.e. 3.5xSFS memory is needed 'by design'?

If so OK, I could see 2xSFS being needed for gapless as in SFS1 holding current track and SFS2 holding next (or, if SFS is small, SFS1 holding current 'piece' and SFS2 holding next 'piece').

But if that's how it works then buffer #1 above just screams to be eliminated as it seems relatively easy to do any conversion as file is being read instead of reading the whole file and then converting the whole thing. Sounds like perfect application for memory-mapped files approach...?
(btw: does 'conversion' mean e.g. x-upsampling/Arc prediction stuff? I'd sure like my WAVs _not_ to be 'converted' as I don't use any of that stuff...)

Come to think of it, do I understand correctly that this 'conversion' is happening _during_ music playback? (as I can see that next track (or 'track piece') is being loaded by 2nd thread during playback of current track ('piece') just a little before it ends)?

Wouldn't it be an idea to instead do this 'conversion' when 'Copy to XX' is ticked in parent process?

Then all file copying/conversions (i.e. really nasty torrent of I/Os & CPU cache-killing memory copies) would be done _before_ music starts playing (assuming 'Start XXEngine during conversion' is not checked).
Wouldn't that mean you'd both need far less memory and have a lot less 'system disturbance' during playback?

Or have I misunderstood completely?

Not sure what is your definition of 'live' but it sure seems 'live' to me as it is owned by XXEngine3 process and definitely taking up RAM space
Check out below those LargeObjects and total stats.

Great!
Although, reading what you wrote today let me reserve some doubt before I see it
Here's why: Looking at current version, I noticed rather strange memory pattern usage and I'm wondering if you got rid of it completely in next version?

For example, if I set SFS to 100MB I (turns out naively) expected to see a 100MB buffer somewhere. Oh yeah, it is there. But there is another 100MB one. Hmmm..., ok.  And then, there is yet another one. And, finally, one more, yet this time 'only' 50MB in size.
So, 100MB SFS setting causes XX to allocate 350MB? Sounds like either a bug or some heavy tricks are going on?

I'm not an expert but looking at this it seems rather simple to implement:
http://www.developer.com/net/article.php/3828586/Using-Memory-Mapped-Files-in-NET-40.htm

That does not sound right: when MMCS is active it is 'on' all the time music plays (and it definitely should _not_ be on at all with Engine#4) - How do you determine it's only 'periodically popping up'?