x264/x265 Video Compression & Conversion

[JaredX7]

This topic is to provide useful information about x264 compression, encoding, converting and rendering. Please feel free to post your findings, tests and corrections in this topic.

Subme (Subpixel Refinement) & Motion Estimation

• Subme 7+ HEX is slower than Subme 6 + UMH. Too slow for the improvement it delivers.
  
  The x264 devs are sure that "Subme 7 + HEX" gives more improvement over "Subme 6 + HEX" than "Subme 6 + UMH" would give. Therefore Subme 7 is worth the slow down it causes, UMH isn't.
  
  A way faster HEX search with a better quality subme 7 > a lot slower UMH search with a lesser quality subme 6.
  
  
• If a lot of b-frames are used, UMH + subme 6 runs faster than HEX + subme 7.
  
  On regular film samples HEX + subme 7 is ~10% faster than UMH + subme 6, but on Animes or video with a lot of fades, UMH + subme 6 is faster.
  
  
• UMH has a lower profit than subme 6 --> 7
  
  So subme7 and hex is better, since umh doesn't necessarily result in a better psnr/ssim, but slowed down encoding considerably.
  
  
• It is generally pointless to use Subme 11 as it slows down speed with no real world improvements. If raising Subme, you probably shouldn't go any hither than 9.
  
  

  Quality

• If you want something that is as close as possible to a "constant quality" mode, then use CRF mode. Also be aware that with CQ mode you would loose AQ and MB-Tree!
  
  
• CRF 24 is a pretty good starting point, giving good quality decent-sized files. If more quality is required, drop the value. But you probably don't need to go lower than 16. Also try not to go higher than 30.
  
  

  Other

• Analysis "all" is overkill, because the 4x4 partitions usually won't help at all.

[JaredX7]

My Current Commandline for x264 Compression

Commandlines are current as of October 2017.

--keyint 120 --ref 5 --ipratio 1.6 --pbratio 1.5 --colormatrix bt709 --subme 7 --b-adapt 2 --qpmin 17 --qpstep 21 --vbv-init 0.0 --qcomp 0.4 --rc-lookahead 120 --aq-mode 3 --aq-strength 1.5 --nr 1000 --trellis 0 --psy-rd 0.10:0 --stitchable

+ --crf 19.5: 2560x1440 Insanely High Quality (Shadow Warrior 2)
+ --crf 21: 1920x1080 High Quality (Saints Row)
+ --crf 22.5:
+ --crf 23: 2560x1440 Really High Qualty (Saints Row, Steamworld Dig...)
+ --crf 24: 2560x1440 High Quality
+ --crf 25: 2560x1440 Normal-High Quality
+ --crf 26: 2560x1440 Normal Quality (Racing Games)

* --qcomp 0.3: Very simple 2D Platformers/Sidescrollers (such as Mercenary Kings)

Dark, or very high quality games require lower crf values. Lower resolution also requires lower crf values (deduct the value by around 1.0-2.0 normally), when going from 1440p to 1080p to 720p etc etc.

My Current Commandline for x265 Compression (2017-Oct)

--subme 7 --me 1 --keyint 120 --b-adapt 1 --ref 5 --qpmin 17 --qpstep 21 --qcomp 0.5 --rc-lookahead 120 --aq-mode 3 --aq-strength 1.5 --nr 1000 --no-open-gop --colormatrix bt709 --merange 16 --no-rdoq-level --rd 2 --psy-rd 0.10

AVISynth

ConvertToYV12(Matrix="Pc709")
BicubicResize(1920,1080,0,0.50) # Bicubic (Sharp)

ChangeFPS(30)

KillAudio()

ColorYUV(levels="TV->PC")

BicubicResize(2560,1440,0,1.00) # Bicubic (Sharp)

Fix VEGAS Washed Out Renders:
Levels(16,1.0,235,0,255,coring=false)

input_low, gamma, input_high, output_low, output_high,...)

Optimal CRF Values depending on the type of content & other info

Should still look quite decent

Gameboy / Gameboy Colour: 720p - 25 (could be higher)
Super Nintendo: 720p - 27. 1080p - 25.
PlayStation 1: 720p - 24.
DOS: 22.5
PlayStation 2:

[JaredX7]

My own testing with x264 Compression.

AVISynth Script: ConvertToYV24(Matrix="Pc709")
x264: --output-csp i444
Result: MeGUI doesn't crash.

AVISynth Script: ConvertToYV24(Matrix="Pc709")
x264: --output-csp i420
Result: MeGUI crashes.

AVISynth Script: ConvertToYV12(Matrix="Pc709")
x264: --output-csp i420
Result: MeGUI doesn't crash.

i444 is YV24 and i420 is YV12.
i422 is YV16.

Obviously, PC uses the RGB Colourspace, and the closest to that would be i444 / YV24. But not all devices, such as bluray players or phones, and certain programs don't fully support this range. So in this case we must use i420 / YV12. So we must specify the correct Colourspace and make sure AVISynth matches x264.

PC uses Full Range 0-255. So obviously for videos recorded on PC, we need to have (Matrix="Pc709") specified in the AVISynth Script, and --input-range pc as well as --range pc in the x264 Compression Settings. This is to keep the colours, brightness, and gamma as close as possible to what it's supposed to look like.

[JaredX7]

Aspect Ratio

From 960 to 1280:
AddBorders(160,0,160,0,$000000)

From 1440 to 1920:
AddBorders(240,0,240,0,$000000)

Converting to 4:3, and then adding borders to make it 16:9 at 1280x720 again
Spline64Resize(960,720) # Spline64 (Sharp)
AddBorders(160,0,160,0,$000000)


Most PlayStation 2 Games
NTSC Native Resolution: 512x448.
Upper-most resolution for 720p: 848x720

Addborder Code for AVISynth to make it proper 720p:
AddBorders(216,0,216,0,$000000)

Most Super Nintendo Games
Native Resolution: 256x224

Spline64Resize(824,720) # Spline64 (Sharp)
AddBorders(228,0,228,0,$000000)

[PeeJay Bonobo]

NoVa, do you think you can post your x264 settings for streaming? Though I'm certain it's been too long since you've streamed anything.