Skip to main content

Topic: h264 reference encoder with AviSynth input! (Read 25931 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
I've got via an anonymous developer a special modified build of the reference jm 7.3 commandline h264 encoder, which takes an .avs yv12 (DVD-colorspace)  input and of course outputs .avi.

I believe Roberto will be hosting the binary of this in Brazil soon.

Usage of this encoder is simple:
Code: [Select]
lencod.exe -p InputFile = "video.avs" -p OutputFile = "video.avi"


In encoder.cfg file you can see that by default it retains constant 24 quantizer. By changing this file you can adjust encoder options.

The unmodified build only accpets some raw yuv420 file, so this is a big improvement.
You can do the dvd2avi and avs script build phase in Gordian Knot and then feed the .avs to this commanline encoder.
  • Last Edit: 25 November, 2003, 08:02:15 AM by JohnV
Juha Laaksonheimo

  • rjamorim
  • [*][*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #1
Quote
I believe Roberto will be hosting this binary in Brazil soon.


http://pessoal.onda.com.br/rjamorim/lencod...jm73special.zip
  • Last Edit: 25 November, 2003, 07:16:47 AM by rjamorim
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #2
Ok, the encoder is still slow. Unfortunately the package doesn't include any decoder.
I happen to have a working h264 dshow decoder in testing, but it's strictly forbidden to share it to anybody.

I tested the VSS h264 beta decoder which is publicly available, but seems it didn't work. Somebody might want to test with their latest beta decoder though if it works.
  • Last Edit: 25 November, 2003, 07:43:38 AM by JohnV
Juha Laaksonheimo

  • LordofStars
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #3
Perhaps you might be able to share the information about who created the working decoder so we might look forward to and in the proper channels for it.
r3mix zealot.

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #4
Well.. one thing in the encoder. It currently produces fourcc's a264 and A264 (Edit. it should now produce h264 H264).
Hex edit these to h264 and H264 respectively, and it should work with Mplayer.
http://www.mplayerhq.hu/MPlayer/releases/w...gui-preview.zip

The quality of Mplayer h264 playback totally sucks for some reason. My commercial quality decoder filter gives *hugely* better picture.

Anyway, here are few short demo streams:
http://www.hydrogenaudio.org/extra/H264_q2...q24_600kbps.avi (Reso: 720 x 304)
http://www.hydrogenaudio.org/extra/H264_q2...24_1000kbps.avi (Reso: 704 x 288)

These are both made with constant quantizer 24. The first clip just happens to be about 600kbps and the second about 1000kbps. So these are not really targeted to be these bitrates..rather have constant quality.
  • Last Edit: 25 November, 2003, 09:31:55 PM by JohnV
Juha Laaksonheimo

  • idioteque
  • [*][*][*]
h264 reference encoder with AviSynth input!
Reply #5
Quote
The quality of Mplayer h264 playback totally sucks for some reason. My commercial quality decoder filter gives *hugely* better picture.

It's possible that MPlayer may be short-cutting or skipping the in-loop deblocking filter to get faster decoding.
  • Last Edit: 25 November, 2003, 01:33:39 PM by idioteque

  • danchr
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #6
Quote
The quality of Mplayer h264 playback totally sucks for some reason. My commercial quality decoder filter gives *hugely* better picture.

I believe the libavcodec implementation of H.264 doesn't support all the techniques used in it, yet. I'd suggest you contact the FFmpeg developers and hand them a sample in the hope they're willing to give it a try

  • bond
  • [*][*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #7
JohnV
can you post some screenshots to show the quality plz (perhaps some upsized ones if possible  )?
I know, that I know nothing (Socrates)

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #8
Quote
JohnV
can you post some screenshots to show the quality plz (perhaps some upsized ones if possible  )?

I will, if I get the permission, but at the moment I can't. But believe me, the difference is truely *significant*. Considering for example the 600kbps demo stream, the difference is that Mplayer picture has very bad artifacting, while this commercial class dshow decoder's picture is truely great, and I'd probably rate it at least 1000kbps divx/xvid if it was MPEG4.

Also the 1000kbps H264 looks very blocky and bad quality for this bitrate with MPlayer, but it's very nice with this better decoder.
  • Last Edit: 25 November, 2003, 06:03:41 PM by JohnV
Juha Laaksonheimo

  • Bonzi
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #9
JohnV, probably you are encoding with inloop filtering which is why it looks better on the commercial decoder than mplayer.  Mplayer does not support inloop filtering so it can look really bad if it is used.  They should look about the same though if inloop filtering is turned off because the decoder in mplayer I think is largely based on reference code.  BTW, this makes me very very happy since all of my testing with h.264 has involved converting to yuy2 and then encoding this.  I am a little concerned though about how this is stored in avi though since I don't think there is any standard way to do this, all the other encoders that I know only output the raw h.264 bitstream except hdot264 and vss.  So definitely people don't archive stuff to this codec but it is definitely free game to do lots of testing!

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #10
Quote
JohnV, probably you are encoding with inloop filtering which is why it looks better on the commercial decoder than mplayer.  Mplayer does not support inloop filtering so it can look really bad if it is used.

Indeed, yeah.
I had this in encoder.cfg
Code: [Select]
LoopFilterDisable  = 0  # Disable loop filter in slice header (0=Filter, 1=No Filter)


I'll soon encode something with In-Loop Filter disabled and see what is the difference then. Well, it's said here, that in-loop filtering reduces bitrate 5-10%, so at least file size will increase a little with constant quantizer.
  • Last Edit: 25 November, 2003, 09:16:15 PM by JohnV
Juha Laaksonheimo

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #11
Quote
So definitely people don't archive stuff to this codec but it is definitely free game to do lots of testing!

I don't think anybody archives anything with this. With 3Ghz P4 encoding about 2 hour movie takes about 15 days, if I calculated correctly. Though maybe changing some of the encoder options makes it faster. Currently it takes about 7-8 sec per frame...

I was thinking maybe someone like Frank Klemm who has experience on encoder speed optimizations and now says is infact more interested in video, would like to start tweaking a h264 encoder.
I'm at least going to email him.
  • Last Edit: 25 November, 2003, 10:09:30 PM by JohnV
Juha Laaksonheimo

  • Bonzi
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #12
Quote
I don't think anybody archives anything with this. With 3Ghz P4 encoding about 2 hour movie takes about 15 days, if I calculated correctly. Though maybe changing some of the encoder options makes it faster. Currently it takes about 7-8 sec per frame...

Lol, well, you never know but I would hope that nobody would try to.  Certainly it would be great if frank or anyone else could make a fast high quality encoder or even tweak this one so it is faster.  But, personally I hope someone will write an encoder from scratch.  Clearly, this would be no small task.  The interest in h.264 is building quickly, there are many people including some very talented developers that I know of who are interested.  I even heard that certain persons already have been developing an encoder which now is approximately the same speed as VSS h.264 implementation but much better quality of course.  I haven't seen it yet so we will just see what happens...

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #13
I just contacted ffdshow developer Milan Cutka, and if everything goes alright, we should see ffdshow support for h264 avis pretty soon.
He had problems creating streams working with libavcodec using hdot264 and vssh264, but hopefully this will work. At least the files work with Mplayer which should use libavcodec for h264 decoding.
I'm also soon finishing an encode with in-loop filtering disabled. I'll upload it also.

Someone might wonder why b-frames don't work. [edit] see my next message.
  • Last Edit: 26 November, 2003, 12:52:28 PM by JohnV
Juha Laaksonheimo

  • slav!x
  • [*]
h264 reference encoder with AviSynth input!
Reply #14
Hi, because not eveybody have 3ghz pc , this config should be pretty fast

Code: [Select]
# New Input File Format is as follows
# <ParameterName> = <ParameterValue> # Comment
#
# See configfile.h for a list of supported ParameterNames


##########################################################################################
# Files
##########################################################################################
InputFile             = ""     # Input sequence, YUV 4:2:0
InputHeaderLength     = 0      # If the inputfile has a header, state it's length in byte here
FramesToBeEncoded     = 299    # Number of frames to be coded
SourceWidth           = 352    # Image width in Pels, must be multiple of 16
SourceHeight          = 288    # Image height in Pels, must be multiple of 16
TraceFile             = ""
ReconFile             = ""
OutputFile            = "res.avi"


##########################################################################################
# Encoder Control
##########################################################################################

IntraPeriod           = 300 # Period of I-Frames (0=only first)
QPFirstFrame          = 24  # Quant. param for first frame (intra) (0-51)
QPRemainingFrame      = 24  # Quant. param for remaining frames (0-51)
FrameSkip             =  0  # Number of frames to be skipped in input (e.g 2 will code every third frame)
UseHadamard           =  1  # Hadamard transform (0=not used, 1=used)
SearchRange           =  8  # Max search range
NumberReferenceFrames =  1  # Number of previous frames used for inter motion search (1-5)
MbLineIntraUpdate     =  0  # Error robustness(extra intra macro block updates)(0=off, N: One GOB every N frames are intra coded)
RandomIntraMBRefresh  =  0  # Forced intra MBs per picture
InterSearch16x16      =  1  # Inter block search 16x16 (0=disable, 1=enable)
InterSearch16x8       =  1  # Inter block search 16x8  (0=disable, 1=enable)
InterSearch8x16       =  1  # Inter block search  8x16 (0=disable, 1=enable)
InterSearch8x8        =  1  # Inter block search  8x8  (0=disable, 1=enable)
InterSearch8x4        =  1  # Inter block search  8x4  (0=disable, 1=enable)
InterSearch4x8        =  0  # Inter block search  4x8  (0=disable, 1=enable)
InterSearch4x4        =  0  # Inter block search  4x4  (0=disable, 1=enable)

##########################################################################################
# Error Resilience / Slices
##########################################################################################

SliceMode             =  0   # Slice mode (0=off 1=fixed #mb in slice 2=fixed #bytes in slice 3=use callback 4=FMO)
SliceArgument         = 50   # Slice argument (Arguments to modes 1 and 2 above)
num_slice_groups_minus1 = 0  # Number of Slice Groups Minus 1, 0 == no FMO, 1 == two slice groups, etc.
FmoType               =  0   # 0:  Slice interleave, 1: Scatter, 2: fully flexible, data in FmoConfigFileName,
                            # 3:  rectangle defined by FmoTopLeftMB and FmoBottomRightMB,
                            #     (only one rectangular slice group supported currently, i.e. FmoNumSliceGroups = 1)
                            # 4-6:evolving slice groups, FmoNumSliceGroups = 1, the evolving method is defined by
                            #     FmoChangeDirection and FmoChangeRate.
FmoTopLeftMB          = 24   # the top left MB of the rectangular shape for slice groups, MB counted in raster scan order
FmoBottomRightMB      = 74   # the bottom right MB of the rectangular shape for slice groups
FmoChangeDirection    = 1    # 0: box-out clockwise, raster scan or wipe right,
                            # 1: box-out counter clockwise, reverse raster scan or wipe left
FmoChangeRate         = 4    # SLICE_GROUP_CHANGE_RATE minus 1

FmoConfigFileName     = "fmoconf.cfg"   # not yet used, for future fully flexible MBAmaps

UseRedundantSlice     = 0    # 0: not used, 1: one redundant slice used for each slice (other modes not supported yet)

##########################################################################################
# B Frames
##########################################################################################

NumberBFrames         =  0  # Number of B frames inserted (0=not used)  
QPBPicture            =  30 # Quant. param for B frames (0-51)
DirectModeType        =  0  # Direct Mode Type (0:Temporal 1:Spatial)

##########################################################################################
# SP Frames
##########################################################################################

SPPicturePeriodicity  =  0  # SP-Picture Periodicity (0=not used)
QPSPPicture           = 28  # Quant. param of SP-Pictures for Prediction Error (0-51)
QPSP2Picture          = 27  # Quant. param of SP-Pictures for Predicted Blocks (0-51)


##########################################################################################
# Output Control, NALs
##########################################################################################

SymbolMode             =  0  # Symbol mode (Entropy coding method: 0=UVLC, 1=CABAC)
OutFileMode            =  0  # Output file mode, 0:Annex B, 1:RTP
PartitionMode          =  0  # Partition Mode, 0: no DP, 1: 3 Partitions per Slice

##########################################################################################
# Search Range Restriction / RD Optimization
##########################################################################################

RestrictSearchRange  =  0  # restriction for (0: blocks and ref, 1: ref, 2: no restrictions)
RDOptimization       =  0  # rd-optimized mode decision (0:off, 1:on, 2: with losses)
LossRateA            = 10  # expected packet loss rate of the channel for the first partition, only valid if RDOptimization = 2
LossRateB            =  0  # expected packet loss rate of the channel for the second partition, only valid if RDOptimization = 2
LossRateC            =  0  # expected packet loss rate of the channel for the third partition, only valid if RDOptimization = 2
NumberOfDecoders     = 30  # Numbers of decoders used to simulate the channel, only valid if RDOptimization = 2
RestrictRefFrames    =  0  # Doesnt allow reference to areas that have been intra updated in a later frame.

##########################################################################################
# Additional Stuff
#########################################################################################

UseConstrainedIntraPred  =  0  # If 1, Inter pixels are not used for Intra macroblock prediction.
LastFrameNumber          =  0  # Last frame number that have to be coded (0: no effect)
ChangeQPP                = 16  # QP (P-frame) for second part of sequence (0-51)
ChangeQPB                = 18  # QP (B-frame) for second part of sequence (0-51)
ChangeQPStart            =  0  # Frame no. for second part of sequence (0: no second part)
AdditionalReferenceFrame =  0  # Additional ref. frame to check (news_a: 16; news_b,c: 24)

NumberofLeakyBuckets     =  8                      # Number of Leaky Bucket values
LeakyBucketRateFile      =  ""  # File from which encoder derives rate values
LeakyBucketParamFile     =  "" # File where encoder stores leakybucketparams

InterlaceCodingOption    =  0  # (0: frame coding, 1: adaptive frame/field coding, 2:field coding, 3:mb adaptive f/f)

NumberFramesInEnhancementLayerSubSequence  = 0  # number of frames in the Enhanced Scalability Layer(0: no Enhanced Layer)
NumberOfFrameInSecondIGOP                  = 0  # Number of frames to be coded in the second IGOP

WeightedPrediction    = 0   # P picture Weighted Prediction (0=off, 1=explicit mode)  
WeightedBiprediction  = 0   # B picture Weighted Prediciton (0=off, 1=explicit mode,  2=implicit mode)  
StoredBPictures    = 0   # Stored B pictures (0=off, 1=on)

SparePictureOption = 0         # (0: no spare picture info, 1: spare picture available)
SparePictureDetectionThr = 6   # Threshold for spare reference pictures detection
SparePicturePercentageThr = 92    # Threshold for the spare macroblock percentage

PicOrderCntType = 0       # (0: POC mode 0, 1: POC mode 1, 2: POC mode 2)

##########################################################################################
# Loop filter parameters
##########################################################################################
LoopFilterParametersFlag = 0  # Configure loop filter (0=parameter below ingored, 1=parameters sent)
LoopFilterDisable        = 0  # Disable loop filter in slice header (0=Filter, 1=No Filter)
LoopFilterAlphaC0Offset  = -2  # Alpha & C0 offset div. 2, {-6, -5, ... 0, +1, .. +6}
LoopFilterBetaOffset     = -1  # Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}



##########################################################################################
# CABAC context initialization
##########################################################################################
ContextInitMethod   = 0   # Context init (0: fixed, 1: adaptive)
FixedModelNumber    = 0   # model number for fixed decision for inter slices ( 0, 1, or 2 )


btw, setting RDOptimization      =  1 , can decrease bitrate by few percents, and slow down at least twice 

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #15
Quote
Someone might wonder why b-frames don't work. This is most probably because the encoder currently only supports baseline h264 profile, which means no b-frames.

Umm, according to new info I got this was wrong information. The encoder should be able to encode b-frames as well. For example encoding with 1 consequtive b-frame, edit the encoder.cfg values:
Code: [Select]
FrameSkip             =  1  # Number of frames to be skipped in input (e.g 2 will code every third frame)
NumberBFrames         =  1  # Number of B frames inserted (0=not used)

The skip parameter should be applicable for i/p frames only and "skipped" frame will be encoded as b-frame.
This is not 100% confirmed information, and I haven't yet tested this.
Also I don't know if any decoder supports b-frames yet. My dshow decoder doesn't support those yet.

ANyway, this is something which should be tested..
  • Last Edit: 26 November, 2003, 12:53:30 PM by JohnV
Juha Laaksonheimo

  • idioteque
  • [*][*][*]
h264 reference encoder with AviSynth input!
Reply #16
Quote
Also I don't know if any decoder supports b-frames yet. My dshow decoder doesn't support those yet.

The JM7.3 decoder supports B Frames. B Frames have been supported in the JM encoder and decoder at least since 6.1d, if not longer. (6.1d is as far back as I've tested)

I'm curious if anyone has ever gotten the interlace options to work:

Code: [Select]
InterlaceCodingOption    =  0  # (0: frame coding, 1: adaptive frame/field coding, 2:field coding, 3:mb adaptive f/f)
  • Last Edit: 26 November, 2003, 12:57:26 PM by idioteque

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #17
Quote
The JM7.3 decoder supports B Frames. B Frames have been supported in the JM encoder and decoder at least since 6.1d, if not longer. (6.1d is as far back as I've tested)

Right, I was more referring to some more "useful" decoders like libavcodec, which are used by players or dshow.
Juha Laaksonheimo

  • slav!x
  • [*]
h264 reference encoder with AviSynth input!
Reply #18
JohnV can you test, if that your mystical  decoder supports cabac, it's one of best h264 features

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #19
Quote
JohnV can you test, if that your mystical   decoder supports cabac, it's one of best h264 features

Well.. it's a decoder in development. My version doesn't support h264 main profile features quite yet.
  • Last Edit: 26 November, 2003, 02:57:46 PM by JohnV
Juha Laaksonheimo

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #20
Quote
This package contains the official H.264 / AVC Reference Software
encoder (JM 7.5b) with some quick'n'dirty hacks in the input/output
modules so that it can use AVI files as input and output.

The input AVI file must be in YUV 420 format (FOURCC = IYUV, I420 or YV12).

For the output, the hack is very basic, it considers that each frame is
contained in a single slice and therefore embedded in a single nal unit.
This latter is put in a AVI sample.

The parameters nal units are preppended to each IDR frame and sent to the
AVI file in the same sample.

The FOURCC used for the output AVI is A264, since it is what we're using
for our decoder. Feel free to change it to whatever you want.

Please consider this modification as a very quick prototyping tweak and
should not be considered as a reference for carrying H.264/AVC stream in AVI
files. And of course, you're welcome to improve this stuff.


bobololo@h-cube.net

Source download here.
  • Last Edit: 22 January, 2004, 05:15:07 PM by JohnV
Juha Laaksonheimo

  • plonk420
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #21
how does it compare to VP6? i've been mildly impressed with VP6 to say the least, not to mention On2 had some glowing things to say about their encoder on their site...

"In our internal testing, VP6 beat H.264, Windows Media 9 and Real Networks 9 in PSNR comparisons using the standard set of MPEG-2 test clips. The codec looks better than Windows Media 9, shows far fewer motion artifacts than Windows Media 9, and maintains more texture and detail than Real 9 or H.264."

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #22
Quote
how does it compare to VP6? i've been mildly impressed with VP6 to say the least, not to mention On2 had some glowing things to say about their encoder on their site...

"In our internal testing, VP6 beat H.264, Windows Media 9 and Real Networks 9 in PSNR comparisons using the standard set of MPEG-2 test clips. The codec looks better than Windows Media 9, shows far fewer motion artifacts than Windows Media 9, and maintains more texture and detail than Real 9 or H.264."

Umm.. first of all, I don't think they have had any really commercial quality h264 codec, mainly because those are just beginning to develope. So to say at this point already something about "h264-quality" is pretty much the same if someone talked about "mp3 quality" when Xing-old was new...
Further more PSNR comparison is pretty much unreliable..
  • Last Edit: 29 November, 2003, 08:05:33 PM by JohnV
Juha Laaksonheimo

  • plonk420
  • [*][*][*][*]
h264 reference encoder with AviSynth input!
Reply #23
i'm posting a few tests i did (didn't mess too much with the default settings, altho i finally figured out how to do multipass vbr) to alt.binaries.pictures.leek, if any of you are into newsgroups... vp6+aac/aac-he in the MKV container

i was pretty skeptical when i first downloaded it, but was pretty impressed as it put out better results than i ever got with XVID/Divx5 (or what little i knew of those technologies)
  • Last Edit: 29 November, 2003, 09:17:24 PM by plonk420

  • JohnV
  • [*][*][*][*][*]
  • Developer
h264 reference encoder with AviSynth input!
Reply #24
I got permission to post some pics from the commercial h264 decoder output, so I will do so soon.

Also Milan Cutka (ffdshow developer) emailed me that he got h264 decoding working in ffdshow.
  • Last Edit: 29 November, 2003, 09:18:17 PM by JohnV
Juha Laaksonheimo