![\"\"](\"https:\/\/blog.jpegmini.com\/wp-content\/uploads\/2021\/10\/jpegmini_pro_3_icon.png\"){kind=icon}
<\/a><\/figure><\/div>\n\n\n\n<\/p>\n\n\n\n
Video optimization is not new to Beamr, the company that developed JPEGmini. Beamr has released several enterprise products which optimize video files, based on the same technology that powers JPEGmini. These products have been used by operators of video streaming services to reduce their video storage and delivery costs and to improve the streaming experience for their end-users. But now, for the first time, we are adding video optimization capability to JPEGmini, our consumer product line. <\/p>\n\n\n\n
<\/a><\/figure><\/div>\n\n\n\n<\/p>\n\n\n\n
The technology behind the image and video optimization solutions developed by Beamr is very unique: The algorithms are covered by 34 granted international patents, and we were recently awarded the 2021 Technology and Engineering Emmy\u00ae award from the National Academy of Television Arts & Sciences for this technology. <\/p>\n\n\n\n
In this blog post, we\u2019ll explain how the new video optimization feature works under the hood, and what you can expect when using it on your videos. But before we do that, let\u2019s take a look at the image optimization technology first, as this is the basic building block of the technology. After you understand how it works for images, it will be much easier to extend this concept to video optimization.<\/p>\n\n\n\n
How Does JPEGmini Work For Photos?<\/h1>\n\n\n\n
Basically, the way JPEGmini performs its \u201cmagic\u201d of reducing file size without affecting quality, is by removing perceptually redundant information from the image file. This is information that is imperceptible by humans, and there is no point in keeping information that we can\u2019t see… But how do we identify such information, and remove it, while making sure all \u201cessential\u201d information is preserved? This is where the Beamr Quality Measure (BQM) comes into play. <\/p>\n\n\n\n
BQM is a unique quality metric – it\u2019s a mathematical algorithm, which is highly correlated with the way humans perceive image quality. So BQM can tell us with a very high level of accuracy, whether any visible artifacts exist in our optimized images when compared to the original images. BQM looks at various aspects of the difference between the two images: How the pixel values differ between the input and the optimized image, how much blockiness was added, how has the texture changed, etc. These different attributes are analyzed for each of the image \u201ctiles\u201d, which are small segments of the image, and combined into a single \u201cquality score\u201d for that tile. The tile quality score is then used to control the amount of compression applied to that tile.<\/p>\n\n\n\n More information about BQM can be found in Beamr\u2019s blog post<\/a> and podcast episode<\/a>. <\/p>\n\n\n\n JPEGmini relies on BQM in its optimization process. Basically, we take the original JPEG image and apply higher levels of compression to produce several candidate images that are more compressed than the original JPEG. Then we use BQM to compare the quality of the candidate images to the source image. Finally, we determine which is the smallest candidate (in bytes) that does not exhibit any additional visible artifacts when compared to the original JPEG. This is the selected candidate that JPEGmini outputs as the final optimized JPEG. The process is described in the diagram below.<\/p>\n\n\n\n It\u2019s important to note that the \u201cencoding parameters\u201d we use to create the candidate and final JPEG images are not simply an indication of a single \u201cQuality Factor\u201d (QF) number. We actually go much deeper into the JPEG file, and modify a whole set of parameters that is called the \u201cQuantization Matrix\u201d. This set of parameters determine the amount of compression applied to each individual 8×8 pixel block, and we modify this set adaptively, based on the amount of additional compression we wish to apply, and on the characteristics of the source JPEG image. This enables us to create the most compact representation of the image that is possible under the JPEG standard. <\/p>\n\n\n\n As you probably know, JPEGmini is not limited to JPEG images: In JPEGmini 3.0 and beyond, we use a similar technique to create an optimized JPEG from a source HEIC image. In the future, we plan to support additional input and output image formats, including emerging image formats such as AVIF and JPEG XL.<\/p>\n\n\n\n Is video just a series of images? Well, yes and no\u2026 It turns out that video frames have different characteristics than still images when they undergo video compression, so extending JPEGmini\u2019s capabilities to support video files required some adaptations to BQM. For example, when video frames are compressed, it is possible that sharp edges will be \u201csmeared\u201d, and this type of artifact would be visible to the human eye. In addition, in video it is important to preserve not only the quality of each individual frame, but also the consistency of changes between consecutive frames, which is called \u201ctemporal flow\u201d. In order to capture these additional potential artifacts, we enhanced BQM with two more components: An Edge Loss component and a Temporal Consistency component. In addition, when using the video version of BQM, we \u201cpool\u201d together the tile quality scores to create a single quality score for the video frame. <\/p>\n\n\n\n The diagram below shows the structure of the enhanced BQM that we use for video applications. The method we use to optimize video files is similar to the way we optimize image files, we just apply it to each and every video frame. The video is first decoded, and then each video frame is encoded with higher levels of compression to produce one or more candidate video frames. We then use the video version of BQM to compare the quality of each candidate frame with the source video frame, and determine which is the smallest candidate video frame that, according to BQM, still does not have any visible artifacts compared to the source. The selected candidate frame is then output to the encoded video stream. The diagram below describes the optimization process that we apply to video frames. <\/p>\n\n\n\n By applying the optimization process in a closed-loop, which compares the quality of each output frame with the input, we guarantee that the quality of video is preserved during optimization. <\/p>\n\n\n\n Note that other than changing the compression level of the video frames, JPEGmini does not alter any other attribute of the video file. The file format, metadata, audio track and any other tracks in the files remain untouched and are seamlessly copied and inserted into the output file. This ensures that the video files optimized by JPEGmini retain all of the information in the source video files, and they remain compatible with any video playback or editing software that was used on the source files. <\/p>\n\n\n\n While JPEG is the image format used for the vast majority of digital images in the past 20 years, for video the situation is more complicated. AVC (also known as H.264) is indeed the most common video codec used across the industry, from capture to download to streaming. However, there are other formats used in specific applications: For example, high-quality video master files used for production and editing are often stored in Apple\u2019s ProRes format. Web video is delivered from YouTube in VP9 format. And the newest smartphones from Apple and Samsung can also capture videos using HEVC, a more efficient codec than AVC. <\/p>\n\n\n\n Since supporting all of these different video formats in the first version of video for JPEGmini would be a huge task, and we wanted to put this feature in the hands of users as fast as possible, we decided to support only AVC, the most common video format, in the first release of JPEGmini with video. AVC is supported in the most common file formats for stored videos, which are MP4 and MOV (Quicktime). So if your phone or camera captured a video file, it will most likely be in MP4\/AVC or MOV\/AVC formats and supported by JPEGmini. If you see a message that a certain video file is not supported by JPEGmini, it is probably not using AVC as the video format, or MP4 or MOV as the file format.<\/p>\n\n\n\n<\/a><\/figure>\n\n\n\n<\/a><\/figure>\n\n\n\nHow Does JPEGmini Work For Videos?<\/h1>\n\n\n\n
<\/p>\n\n\n\n<\/a><\/figure>\n\n\n\n<\/a><\/figure>\n\n\n\nUsing The New Video Support in JPEGmini<\/h1>\n\n\n\n