-
Notifications
You must be signed in to change notification settings - Fork 363
Add AVS3 audio and video #474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @paulhiggs, thank you so much for patience and for your contribution. Aside from the comments I've already left below, I have some requests:
- Although avs-common.ts is a nice addition, I don't think that should be part of the library. Take
avcC
for example, we only use the primitive js types. Which many libraries would expect to see. If the intent behind the classes inside avs-common.ts is purely for aesthetics then we should remove it. - As for the BitBuffer, we should either incorportate it inside DataStream class or make it accessible through DataStream. I'm just thinking out loud here but maybe we can call
getBitReader
and access the methods of BitReader from there. BitReader (or BitBuffer as you named) can request bytes from DataStream. What do you think?
demo/boxHtmlTable.js
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we just use toString
instead of toHTML
. I know it's not pretty but until we have a better solution, we should keep everything consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HTML formatting of semantic value is removed in aefb5bd
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh wait, I was just expecting you would rename toHTML
to toString
as we return html for some toString
calls already.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I had assumed that toString() gives unformatted output that could be used in testual reports and possibly post formatted into HTML, MD etc. In other projects toString()
and toHTML()
are kept seperate for that reason with toString() always being available and
toHTML()being an optional enhancement. I will look for instances where you have
toString()` returning HTML and consider that approach
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know, it's not perfect. I imagine, once (if) we have a more modern file reader app we could rewrite some of these toString
conversions so that it's much more streamlined.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am happy to leave it in string format for now - it is still readable and allows the MP4 file to be inspected. If formal toHTML()
support is added in the future, I can add it into the AVS3 related boxes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we are ever going to add toHTML
or possibly even toString
. If we revise the file reader demo, I would expect display related stuff would be there.
Right now the important thing is that how these boxes look like in the current file reader demo. If it would be better to display some fields as a html table for example, toString would need to return that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The output without HTML is OK.
private endianness: Endianness; | ||
private _buffer: Array<number>; | ||
private _state: State; | ||
private _big_endian = true; // results are returned Big Endian |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this if we have endianness
? If we access BitBuffer (or BitReader) from DataStream, we could use the endianness from there as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, its possible to use the endianess from DataStream, but at present DataStream does not provide reading and writing of arbitrary bitlength values. Sine the structure header informs on the total number of bytes needed, it seems easier to read these into a bit based reader/writer with a seperate deserialize/serialize process into the DataStream.
If there are other MP4 boxes that would benefit from bit level read/write then we could consider adapting DataStream in the future - just don't want to be disruptive at this time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Except for sample entries I don't think there are many boxes that would benefit from this. But in the future we might process samples to get codec specific information. At that time it could be useful to have this functionality available through the DataStream.
I'm okay with keeping BitBuffer class seperate as long as we can access it through DataStream. We could switch to bit level reading on-demand if needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm okay with keeping BitBuffer class seperate as long as we can access it through DataStream. We could switch to bit level reading on-demand if needed.
I started looking at this but felt that for the interim it was better to have a seperate bit-wise parser and not interfere with DataStream that is relied on by many classes. Some other nuances can exist as well, for example with boxes are not byte aligned (happens in transport streams with descriptors but perhaps not in ISO BMFF boxes)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess by the time you've started this PR (original one #456), we didn't had an easy way to test everything. As long as the tests pass on this PR (or run npm test
) it should be all good.
On gpac, we have Bitstream namespace that handle both byte and bit level parsing: https://doxygen.gpac.io/group__bs__grp.html
gf_bs_read_int
is all that was needed for bit level reading. Excluding the emulation prevention byte handling of course. We don't need that much stuff right now though.
If you don't want to combine BitBuffer with DataStream, or make it accesible from DataStream. We should at least make it possible to construct BitBuffer via DataStream. For example:
const ds = new DataStream(buffer)
const bs = BitBuffer.from(ds)
// Would need to handle already read bits when we use ds again
Or something like that. Just so that we won't need to copy the data beforehand. When we drop/discard the BitBuffer the state in DataStream would stay intact (the same one given to box parsers via the parse()
method).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was initially considering such an approach, but since for the current AVS3 case I knew how many bytes I would need, it was easier to grab them from DataStream and reuse a port of BitBuffer from C++ that was used elsewhere. Certainly, when an infinate (or extremely long) bitstream is needed to be parsed in a bit-by-bit manner, the approach you identify makes the most sense and when the bit reading is finished, the next byte ready would pick up an aligned 8 bits from DataStream.
Seperately, I will look into this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Precisely. I guess having this available via or compatible with DataStream
would make it a very useful contribution.
On the classes defined in avs-common.ts: I took this approach as the semantif definition (and to a degree the representation of th value as binary, decimal, hexadecimal) is guided in the parsing process, so its the best place to determine how the output should be formatted while still retaining the possibility to ready the original value back for future use in |
Yes, they are for aestetic purposes - describing the semantic value of the field in the box. If these are not of interest to the project, I will remove them. |
They could be useful in the filereader demo, but probably not so much useful for the users of this library. As those libraries would be much more interested in accesing the values of these fields directly rather than the human-readable format of the field. That's my opinion though. I just don't want to complicate it more than necessary. |
…in line with other box displays
Description
The PR adds support for the relevant sample types for AVS3 audio and video.
AVS3 video files are available at DVB - https://dvb.org/specifications/verification-validation/avs3-test-content/
AVS3 audio files are also available at DVB - https://dvb.org/avs3-audio-test-content/
See also #456