Interview with PS3 system software engineers

[gunnerhkjp]

A little by-product of my Japanese language practice :laugh:

http://www.watch.impress.co.jp/av/docs/20070614/rt033.htm

Talking to PS3 system software engineers about leading up to "1.80" and its future.

The 1.80 software released on May 24th tansformed the PS3 as an AV equipment.

The PS3 gained high quality DVD upconvert using the power of the Cell processor, DLNA and remote play, and network enhancements.

Even with the past experience of PSPs, not many people expected the PS3 to change so drastically with a software update.

Here is an interview with the PS3 software engineers, SCE CTO Kawanishi (GM software platform development) and Kanehide (Manager in charge of AV development)

Q. Upconversion was added half a year after the release. Please discribe the process leading up top it.

Kanehide (K2)

Sorry to keep you waiting for half a year.

Kawanishi (K1)

I said this in past interviews, but if we did a normal bilinear upconversion, then the picture quality would just be the same as what is normally done by TVs and computers. Since we had the Cell, we thought we had to do better than that.

Taste factors greatly in the quality of upconversion, so we had to have various critics analyze the picture quality. That is why we couldn't get the feature out in time of the release.

K2

I think we finally got it to a level that is not enbarassing as a product. We have gotten good reviews from critics, but it was tough considering the critics work with AV equipment that cost the same as 100 PS3s.

We have gotten feedback from them regarding several failure patters for upconverting, but we are getting closer to the goal. I think we can improve another 30 to 40%.

K1

And there's still power left in the cell processor.

Q

I have heard that the upconversion was specifically designed with anime in mind. Why is that?

K2

The upconversion is not specifically tuned for anime contents. It works just as well with CG, real life pictures and movies.

The misconception probably came from the fact that we used anime contents often to check for IP conversion errors.

Upconversion quality does not improve without IP conversion tuning, which is a lot easier to do with anime then natural pictures.

Its more acurate to say that tuning was done extensively with anime for our convienience, but quality analysis was done with various kinds of content.

Upconversion works better with content with high SN ratio. Anime is a good example of such content, but it varies with bitrate, and also if the content was created digitally or from old film cells.

The quality increase that will come from now on will probably depend more on the application of noise reduction than the quality of upconverison.

Q

An example of content with bad SN ratio is TV recording on DVD recorders. It seems the upconverion doesn't work as well on these contents.

K2

I agree. Quality for normal DVD recorder content with 4Mbps Mpeg2 will depent greatly on the quality of noise reduction.

Ofcourse, it cannot be perfect. The problem is how much noise to remove. Currently, the NR parameter has 3 levels, so a stronger NR on such contents may get better results.

The most difficult part of programming the upconverion was the routine for NR judgement to distiguish between noise and detail.

Q

What is the benefit of NR using the cell processor.

K2

The benefit is that NR can be melded into the entire process of picture processing, as opposed to it being a separate process within it.

For example, if we judge that a certain part of the picture is noise or artifact, we can choose not to upconvert that part. With dedicated chips, this is hard to do.

Also, we have put in the NR at various points for the processing, instead of applying it all at the end. For example, NR for common DVD players is applied AFTER the subtitles are inserted. We have have the NR applied before the subtitle, so that it appears clearer.

PS3 also contains many dedicated chips as well, but its application can be changed with flexibility. Higher picture quality can be achieved by tweaking the usage.

Q.

What improvements can we expect?

K2

I think we are currently making improvements that is easy to see. I expect much of the upcoming work to be a lot less visible, but never the less important. Improvement in the NR process is one such improvement.

K1

A lot of attention is put in the upconversion quality, but as HD contents become the norm, then picture enhancement for those contents will become more important.

Q.

How about upconversion of 1080i BD titles to 1080p?

K2

We are currently assesing such feature internally, but it is difficult to do until the Blu-ray spec is finalized. Blu-ray spec is always being updated, and we have to make sure that there is enough hardware resource left to accomodate the updates.

Even if we could do various updates and addition of functions now, we have to avoid using up the resources which may lead to us taking away such updates to accomodate spec updates.

Q.

Does that mean you have to leave some processing power and memory for spec updates like network functions?

K2

Maybe. Even if we could implement 1080i to 1080p upconversion, which is VERY heavy on the processor, the level of quality will have to be tuned.

Q.

Maybe the upconversion does not need to be done in real time. How about high quality upconversion by investing longer processing time?

K1

That's in the region of re-mastering. Currently, real time processing is the norm.

K2

It is one of the possibility that we are assesing, but I feel that the strengh of the Cell processor lies in real time processing.

I guess we could get superior quality by letting the machine work on it over night. For now, we want to concentrate on real time.

There is a limit to how much can be achieved real time, and deferred processing is a viable alternative. But such high end quality enhancements are hard to see by normal people.

But if there is a need, we will do it.

Q.

Regarding sound. I think nobody expected the SACD function of the PS3 to be so complete.

K2

We didn't intend it to be such a featured function, but we decieded that it would be one of the things we concentrate initially to get a handle on the Cell processor.

We were only able to do 24bit/88.2kHz degimation, but got it up to 24bit/176.4kHz. We worked the the department which makes Amps to really get a feel for how sound is processed. When SACD is playing, cell had nothing much to do. So we concentrated on how to use the cell to improve the quality of the sound further.

The know how we got from the process for immense, although it took us away from the the realm of playstation and into the world of high end audio.

Q.

Looking at eh SACD playback, it seems emphasis was placed on sending PCM over HDMI more than the quality of the analog outputs.

K2.

The reason for that is simple. Although the analog output quality of the PS3 is very high, it really can't win against AV amps costing thousands of dollars. Where we can compete in is the processing power of the Cell. So we concentrated of passing good PCM data to the amps through HDMI.

PS3 cell processor is a lot better than the chips found in AV amps, so we figured we will do all the processing on the PS3, and pass the result to the amps, getting much higher quality. Ofcourse, the only connection that can pass sound data at that quality turned out to be HDMI.

Q.

How about optical out?

K2

Optical out does not have the copy proctection mechanism necessary for SACD output. Its unfortunate.

Q.

How out CD upsampling?

K2

Its being worked on. CD don't have the restriction on optical out, so it will be easier to use.

Q.

What other functions are you working on? How about virtual surround for 2ch stereo?

K2

That would be interesting. We are trying to figure out what people want.

For example, what was done before by AV amps can pretty much be done by the PS3. So functions like virtual surround can be added to accomodate people who do not own an AV amp.

Q.

There is a lot of demand for a jukebox function. How about functions like lossless transfer into the PS3 and album shuffle? Also, how do you see the link with portable players?

K1

We haven't really decided on the positioning of the PS3 as a music player. We don't know how much people want a music player that requires you to turn on the TV, for example. SACD really pushed the PS3 in terms of audio, but we have to guage user demand.

But Sony produced many products, and I could forsee the PS3 becoming a "mother ship" to all the equipment.

Ps3 really is a feature rich player, but the basis for all this is that it is a game console. (laugh)

Q.

Linking to games, the 360 has a feature to allow music in the HDD and network to be played back as game BGM.

K1

PS3 already has such function in the system software. It is up to the game developers to use the function. Same goes for pictures and videos.

Q.

The upscaling routine is different for BD/DVD, video files, and games?

K2

Yes

K1

The games require a lot more processing power, so less is left for upscaling.

Q.

Right now, output from the GS (PS2 graphics chip) is upconverted. Can the GS output higher resolution by itself?

K1

Currently, the upconverion is done at the final stage of the output. For PS2 software, it would be difficult to increase the resolution natively. PS1 is a different matter, since it is software emulation.

Q. Why only client for DLNA? I think PS3 should have a server function.

K1

The primary reason is that PS3 does not have a tuner.

Another reason is that consumers are not yet familiar with DLNA, and emphasizing the server function may lead to more confusion.

I personally feel that DLNA has a lot of restriction that may make it hard to use. I think the easiest way is to allow network browsing of files by using samba, for example.

People who are using the DLNA function right now are people with high PC literacy. So we tried to make the PS3 DLNA clinet as compatible as possible with various PC DLNA servers. We also tested with various NASs with server functionality.

Q.

One thing about video file playback. It seems that quality is not constant, and picutre quality falls from time to time.

K1

We don't have an answer to how much quality we can guarantee from playback of video files created by PCs. Although there is standards such as "mpeg-4", there are many cases where the files contain data that are way off the standard.

K2

We have codecs in place that play back data following proper standards, and functions to improve the quality of these picutres. We need more work on handling data that is outside the standard.

K1

For now, out main goal is to allow playback of as many standard as possible.

Q.

From that point of view, how about playback of WMV and Divx?

K1

The problem with those formats are that they are not industry standards like "mpeg-4", but format that is locked in by various corporations.

But I understand the demand (laugh).

Q.

How about overcoming the licencing fees for such formats by selling update for WMV playback at the playstation store for 500yen?

K1

We don't want to resort to such measures (laugh). And its not a matter of not implementing because it is by another company.

Q.

Your intention for remote play is to make people play outside the home?

K1

Yes. There are SDKs out to allow game developers to integrate remote play into their games, so it is a matter of time before such games arrive.

However, it is now suitable for all types of games. There is a certain amount of latecy involved.

Q.

The output has to be handled by SDKs? How about a simple frame buffer transfer with encoding to allow any games to be played this way?

K1

To be honest, that's how it works behind the SDK. The reason why SDK is needed is because the encoding requires processing power that may or may not be left over from the game. Therefore, we cannot guarantee that it will work for all games.

K2.

On top of that, the controller button configuration is different too.

Q.

The biggest problem with the use of remote play right now is that the PS3 has to remain turned on when you leave the house. Is there a solution to this?

K1

Wait and see (laugh).

Q.

How about allowing usage on other than PSP?

K1

Good question. Oficially, no comment. But if you can imagine, it should be possible on any terminal. The only problem is support and service quality.

Q.

What is the current system software development span?

K1

We try to get one out every quarter.

Q.

For how long?

K1

The scary thing about the PS3 is that we can continue to add updates as long as there is space on the HDD (laugh)

We won't stop though, since adding new functionality is a lot of fun. We really enjoy getting feedback from the customers, and finding thing we really didn't think of.

Q.

How do you position the PS3 with other home appliances? I feel sorry for the other AV devices that don't have the benefit of powerful processor and constant updates. Especially inside Sony, you have MANY rivals within the same company.

K1.

We are in difficult position sometimes, but we don't ristrict what we do. If we do, we are finished as engineers.

K2

Also, we don't have the benefit of releasing a new model every 2,3 years. So even if we are at an advantage now, they will catch up with ease if we don't improve.

K1

Ofcourse, we will eventually reach the limit with the current Cell processor. There is a limit to how much one machine can do. I hope we can keep on going until we pass the torch to the next generation gracefully.

Q.

Has anything changed now that Mr. Kutaragi is gone?

K1

No.

Q.

The reason I ask is that many people think that good AV centric updates like the 1.80 would now be over with Mr. Kutaragi gone. Mr. Kutaragi was a REAL audio visual equipment "geek" and without him the priority may be different.

K2

I can see why you would ask that question. The timeing (of Mr. Kutaragi's change and update release) is impeccable (laugh)

It is all a coincidence.

K1

Sony view towards AV technology can never change. It is the foundation of what we do.

Business choices may change, but the foundation is there.

Q.

PS3 is a very powerful hardware, but as more processing power is used, electricity consumption and fan noise increases.

For AV equipment, noise is a big factor.

K2

So far, we are not worried too much since we have plenty of processing power left even at peak usage.

On the other hand, from a software programming point of view, we have arrived to a stage where things work as they should.

But there are plenty of optimization left to do. We have to balance function with load and noise.

K1

Nothing yet comes close to the load that Folding@Home applies. And I can see the possiblity of a fanless PS3 in the future, just like there was for the

PS2.

Q.

Which areas do you think the PS3 has the most chance of growth?

K1

For Audio and Visual function, it will evolve in the current route. PS3 has many input / output capability, and processing the input for various use will be interesting.

K2

There are other functions for Audio and Visual that are upcoming. Wait and see.

[iamthelaw]

thanks interesting read

[dnast]

Thanks for translating. I didn't feel like reading it at first, but it was a good read.

[an_idiot]

wow thx for translating!!!!!!!!!!!!!!!!! great interview!

[Meshuggah]

This made Kotaku

http://kotaku.com/gaming/av-club/ps3-softw...ture-270327.php

Good read.

[troist]

that was long but very interesting. its a bit of a shame that they aren't going to include divx/xvid in the future because they aren't "industry standards" :(

thanks a lot for translating! that is a long passage to translate

[gunnerhkjp]

Glad to see that there is demand for interviews such as this.

It takes a BLOODY long time to translate, even though I was going for speed over accuracy.

And ideally, Sony PR should be doing this :laugh:

There are many interviews at the website which gives people a peek at the real stuff behind the marketing machine for both the PS3 and the 360. This particular interview caught my eye since it showed Sony in the light of an AV equipment manufacturer.

I'm surprised it got mentioned at other sites.

I better watch my grammer and spelling next time :blush:

<Addition>

Upconvert with PS3

Upconvert with TV

Edited June 20, 2007 by gunnerhkjp

[Holmen]

Thanks for the translation, really intresting to read about.

Q.

Right now, output from the GS (PS2 graphics chip) is upconverted. Can the GS output higher resolution by itself?

K1

Currently, the upconverion is done at the final stage of the output. For PS2 software, it would be difficult to increase the resolution natively. PS1 is a different matter, since it is software emulation.

That one makes it intresting for us europeans, perhaps it was a good move to make the PS2 games software emulated in the end?

[Huleboeren]

Keep the translated interviews coming, gunnerhkjp - good reads!

Thank you