Marco’s blog

The intra frame coding of the encoder works. This doesn’t mean it is perfect. The encoder can only use the LeGall wavelet. Quantization is not being used, so the files will be rather big. Also the LeGall wavelet is not ideal for intra frames, so I will add the Deslauriers-Debuc wavelet soon. The encoder currently just does whatever is required to produce a Dirac compatible bitstream and nothing more. The results are playable with my decoder, but unfortunately not with the reference implementation. The reference implementation does not crash or so, it just outputs garbage, like you can see below.

Now I will work on regression tests, I will try to get something to make testing easier into subversion soon. Using this, I will fix any regressions that were introduced last week.

After the decoder was finished, I got some feedback on my code and worked hard to improve my code where possible. These were mainly small changes, but a lot of small changes are still a lot of work. Because I was quite annoyed by the bad quality of the motion compensation code and how slow it is, I worked a bit on improving this. I mainly merged the MC code with the qpel/eighthpel code. This made it possible to eliminate certain checks for border conditions. It also made it possible to avoid some multiplications. This resulted in a 50%(!!) speed-up of the MC code.

In the meanwhile I worked on the encoder. First I implemented Golomb coding and arithmetic coding. After that I started working on writing the Access Unit Header. A the moment I am working on packing of coefficients. First I will just write the pixels from the frame to encode directly to the file, without doing a DWT first. This will result in big pictures that will not be usable in practice because this is not support by the standard. But this is a simple way to test my current code actually works, before I start working on the DWT algorithms. After implementing a DWT (even a simple one like Haar) will make it possible to encode files that can be played by any Dirac decoder. These files will just be very big

Everyone remembers the day on which the statue of Saddam Hussein was pulled down by massive amounts of Iraqis. It seemed lots of people in Iraq were happy. They were the ones that pulled down the statue, it was their choice and they were able to do this and gain their freedom. The USA just gave them the chance to do this, nothing more.

Right…? Wrong! Although this is (more or less) what I thought what was going on. Of course I expected it wouldn’t be exactly like it was shown to us by the media, but it was almost the opposite of this! Perhaps I am an idiot for falling for this, or perhaps I should have payed more attention. Yesterday, in the TV broadcast of Een Vandaag, the photographer Geert van Kesteren said this was all fake. When you zoomed out from the square you would see that there were only about a hundred Iraqis or so. This was just a nice media show organized by the united states goverment.

Of course I did some searching on the Internet as well and I found a website that covers this event and the lies surrounding it. Something that shows how untrustworthy Bush is, is this quote from a CNN transcript “BUSH: I don’t think I’ll ever forget, sure a lot of other people will never forget, the statue of Saddam Hussein falling. Baghdad, and then seeing the jubilation on the faces of ordinary Iraqis.”. He sets up the lies (just like when his friend Colin Powell discovered some interesting use of photoshop) and uses it as evidence. Although it scares me more when you hear how many people still believe the lies about the evidence that there were “Weapons Of Mass Destruction”.

There were some bugs in the decoder. Although the decoder worked, there were still a few bugs and regressions. These have all been fixed. One annoying bug was that the weighting code was incorrect. The main reason for this is that the specification was not correct at this point. At the end I just dumped the spatial weighting matrix from the reference decoder and figured out by hand how it should be calculated. The main problem here was related to rounding. One bug was related to calculating incorrect MC block ranges. The pseudo code in the specification was wrong, so my code was also wrong. Besides that, there were some off-by-one errors, typos, etc.

In the meanwhile I optimized my code significantly. The motion compensation code was completely restructured. This resulted in a 2-3 times performance boost of this code. The IDWT code got a 2x performance boost.

So now the video plays perfectly, the output is exactly the same as the output of the reference implementation. It is still quite slow, but certainly not as slow as it used to be. Because I restructured the code, I think MC can be optimized even further. The most time consuming code is the interpolation code, which can be easily vectorized.

I don’t think I have to say I am really happy with these results :-). Instead of working on the optimizations and vectorizing, I first will work on the encoder. It’s easy for other people to optimize my code, they have more experience. I know quite a lot about Dirac and my code so I can better start working on the encoder.

Last Friday I went to a cafe to eat. On their menu I noticed the weirdest thing on their menu that I have ever seen on a menu: deep fried Snickers.

You can see the entire menu on the website of Cafe Crash. The food I had was good and really cheap. After searching with Google I noticed that deep fried Snickers are being sold at other places too, I even found a recipe!

For a few days I have not been writing about the progress on my Summer of Code project. This is actually good news because I was really busy working on it! The last days were spent working on optimizations. I have been able to speed up my code and will write about that soon when the final optimizations are finished.

Finally the decoder works! Today I fixed some bugs in the interpolation code. After that the frames were played back, but not yet in the correct order. After reordering, the video plays back quite well! There are some visual distortions at the border of the screen I have to look into. There are some other distortions that are caused by the encoder. Most likely because I used an old encoder to encode this file. I have reencoded a few second of the output of my decoder to MPEG so you can have an impression of what the output of the decoder looks like at the moment. I must add to this that the decoder can not decode at full speed yet.

Dirac video reencoded to MPEG

Now I am first going to work on the visual problems and check the output of my decoder to the output of the reference implementation. After that I will update the code so it will work with Dirac files as described by the current specification (from CVS). After some more testing, I will make some obvious optimizations.

The most important optimization I want to make is in motion compensation. At the moment I am looping over all the pixels and apply the blocks to be motion compensated. This is how it was described by the specification, I wanted to have this working first, besides that I had no previous experience with motion compensation. Now I noticed that looping over the blocks is way more efficient. You can apply the same options for a big set of pixels and it is more natural. It appears that a lot of decoders work this way. I wonder what increase of performance will be.

When I moved the code to remove the reference frames to after the decoding process, I used a buffer to keep track of the indexes of the frames to remove in the reference frame buffer. Of course when removing one frame, the indexes of all frames after this one change. Because of this, the incorrect frame was removed. This was a silly, small and simple bug. Unfortunately it consumed quite some time to find. This bug is fixed now.

At the moment I am figuring out how to proceed with the interpolation code. The current code does play back the video, but with a lot of artifacts and sometimes the entire screen is messed up. In the meanwhile, I am trying to find a bug in dirac_get_se_golomb. For some reason, it does not return the correct sign on the PPC.

So I didn’t really do exciting things today, but not everyday can be as productive as I hoped it would be. At least some problems are now fixed and the PPC bug is localized.

Today I have been working on motion compensation itself. For now I try to make my code resemble the specification. It is terribly slow, but the good thing is that I know where it can be improved when it works. The bad thing is that it doesn’t work yet, but that is mainly because it is not fully implemented yet.

Today I noticed that the bugs I reported to the BBC were fixed in the specification as it is maintained in CVS. Today I reported some more bugs. The main one was that the specification said I have to remove retired reference frames before decoding a frame. This was wrong, it has to be done after decoding a frame. My problem was that a reference frame that was needed was removed this way.

I actually spend quite some time at looking at motion compensation in FFmpeg, especially looking at how to deal with reference frames. At the end I ended up implementing this on my own so I am sure the rules imposed by the Dirac specification are followed. Michael reviewed my patch and didn’t comment on how it worked, so I assume it is ok.

Now I have to work on getting the final bits in place and debug my code whenever it is necessary. I am looking forwards to getting this to work and to optimize my code so it will run faster and more efficient.

In #grub one of the most asked questions is if GRUB can boot iso9660 images. Or better: how GRUB should do this, people don’t even think about if it would be possible or not. Of course this annoys many people and “GRUB can not boot CDROM images” is nowadays the most important line in the topic.

Actually, GRUB can read ISO9660(”iso”) images. It can for example loads the first few sectors and boot it. But most people do not realize is “what then?”. What would the loaded operating system do? It will most likely look for a CDROM, which it won’t find and fail.

A while ago I wrote a “loopback” command for GRUB 2. It makes it possible to load files from filesystem images. Think for example of a kernel and an initrd. But even this is not enough. The loaded kernel is not able to access the ISO9660 image just like that. The operating system needs support to deal with this. Somehow you need to pass the path of the ISO9660 image to the kernel and the kernel should loopback mount this image and use it.

So in order to use it I need an operating system that supports this. Sidux is one of those operating systems that do. The advantage of using GRUB 2 is that you do not need to extract the kernel+initrd manually, but can do this at boot time. So all you need is the ISO9660 image.

I managed to boot Sidux using GRUB 2 and the following commands (after renaming the .iso file):

loopback loop (hd0,7)/sidux.iso
linux (loop)/boot/vmlinuz boot=fll quiet vga=791 fromiso=/sidux.iso
initrd (loop)/boot/miniroot.gz
boot

This works perfectly and all you need is GRUB 2 and an the file
sidux-2007-02-200705281503-tartaros-kde-full.iso

I of course tried to do the same with Knoppix. It has a bootfrom option, but according to some forum post I found it doesn’t work when not booting from a CDROM. So I would have to wait until knoppix is fixed before this would work. I also hope Debian, Red Hat, etc will eventually make this possible.

What would be even better is that all CDROMs come with a multiboot bootloader as a file on the CDROM. For example /multiboot-IA32 for a PC CDROM (suggested by Robert Millan on #grub). This will make it possible to load some multiboot (multiboot2 perhaps?) kernel from the CDROM or CDROM image. That kernel (which can also be GRUB 2) will load a grub.cfg specific for the CDROM and boot it. To be able to load a ISO9660 image the multiboot-IA32 bootloader will need the path. The advantage of multiboot is that you can pass this path to the bootloader. If all distributions agree on this filename and agree on the argument to pass to the kernel to be “iso=”, it would be easy to boot such CDROMs without any legacy BIOS support, we can use ISO9660 images and perhaps even LinuxBIOS+GRUB2. Your firmware doesn’t even need CDROM support in the long term, GRUB2 just needs it and will get this support in the short term. And this can even be used on other architectures. Please let me know if this sounds like a good idea that will be supported. Perhaps we just need to create a new CDROM boot “standard” out of this :-).

(more…)