User:AndreyAkifev/sandbox

This is the user sandbox of AndreyAkifev. A user sandbox is a subpage of the user's user page. It serves as a testing spot and page development space for the user and is not an encyclopedia article. Create or edit your own sandbox here.

Other sandboxes: Main sandbox | Template sandbox

Finished writing a draft article? Are you ready to request review of it by an experienced editor for possible inclusion in Wikipedia? Submit your draft for review!

Frame Rate Up-Conversion[edit]

Frame rate up-conversion is the process of increasing the temporal resolution of a video sequence by synthesizing one or more intermediate frames between two consecutive frames. A low frame rate causes aliasing, yields abrupt motion artifacts, and degrades the video quality. Consequently, the temporal resolution is an important factor affecting video quality. Algorithms for FRC are widely used in applications, including visual quality enhancement, video compression and slow-motion video generation.

Methods[edit]

Most FRC methods can be categorized into optical flow or kernel-based^[1]^[2] and pixel hallucination-based methods.^[3]^[4]

Flow-based FRC[edit]

Flow-based methods linearly combines predicted optical flows between two input frames to approximate flows from the target intermediate frame to the input frames. They also propose flow reversal (projection) for more accurate image warping. Moreover, there are algorithms that gives different weights of overlapped flow vectors depending on the object depth of the scene via a flow projection layer.

Pixel Hallucination-based FRC[edit]

Pixel Hallucination-based methods use deformable convolution to the center frame generator by replacing optical flows with offset vectors. There are algorithms that also interpolates middle frames with the help of deformable convolution in the feature domain. However, since these methods directly hallucinate pixels unlike the flow-based FRC methods, the predicted frames tend to be blurry when fast-moving objects are present.

Instruments[edit]

Tool	Аvailability	Maximum frame increase multiplier
Adobe Premiere Pro	Commercial, 7-day free trial	100
Vegas Pro	Commercial, 30-day free trial	100
AviSynth MSU Frame Rate Conversion Filter	Commercial	Any positive integer number
Advanced Frame Rate Converter (AFRC)	Free	Any positive integer number
Topaz Video Enhance AI	Commercial, 30-day free trial	100

Adobe Premiere Pro - Adobe Premiere Pro is a commercial video editing software program that allows you to slow down your video using optical flow and time remapping effects to conventionally shot footage to create better looking and smoother slow motion.

Vegas Pro - Vegas Pro also is a commercial video editing software program. There is a method to make slow motion video too. To perform it you need to choose the motion magnitude in your video and percentages of playback speed.

AviSynth MSU Frame Rate Conversion Filter - The AviSynth MSU Frame Rate Conversion Filter is an open-source tool intended for video frame rate up-conversion. It increases the frame rate integer times. It allows, for example, to convert a video with 15 fps into a video with 30 fps.

Advanced Frame Rate Converter (AFRC) - Main advantage of AFRC algorithm is using of several quality enhancement techniques such as adaptive artifact masking, black stripe processing and occlusion tracking:
- adaptive artifact masking technique allows to make artifacts less noticeable for eyes thus increasing the integral quality of processed video;
- black stripe processing allows to avoid artifacts which are commonly appeared in interpolated frames in case of black stripe presented near frame edges;
- occlusion tracking performs high quality restoration of interpolated frames near edges in case of presence of motion with direction to/from the frame edge.

Topaz Video Enhance AI - Topaz Video Enhance AI has the Chronos AI model uses deep learning to increase video frame rate without artifacts. This algorithm generates new frames that are often indistinguishable from frames captured in-camera.

^ Simon, Niklaus; Long, Mai; Feng, Liu (2017). Video frame interpolation via adaptive separable convolution. ICCV. arXiv:1708.01692.
^ Huaizu, Jiang; Deqing, Sun; Varun, Jampani; Ming-Hsuan, Yang; Erik, Learned-Miller; Jan, Kautz (2018). Super slomo: High quality estimation of multiple intermediate frames for video interpolation. ICCV. arXiv:1712.00080.
^ Shurui, Gui; Chaoyue, Wang; Qihua, Chen; Dacheng, Tao (2020). Featureflow: Robust video interpolation via structure-to-texture generation. IEEE. doi:10.1109/CVPR42600.2020.01402. ISBN 978-1-7281-7169-2.
^ Myungsub, Choi; Heewon, Kim; Bohyung, Han; Ning, Xu; Kyoung, Mu Lee (2020). Channel attention is all you need for video frame interpolation. AAAI. doi:10.1609/aaai.v34i07.6693.

[Simon_Niklaus,_Long_Mai,_and_Feng_Liu-1] Simon, Niklaus; Long, Mai; Feng, Liu (2017). Video frame interpolation via adaptive separable convolution. ICCV. arXiv:1708.01692.

[Huaizu_Jiang,_Deqing_Sun,_Varun_Jampani,_Ming-Hsuan_Yang,_Erik_Learned-Miller,_and_Jan_Kautz-2] Huaizu, Jiang; Deqing, Sun; Varun, Jampani; Ming-Hsuan, Yang; Erik, Learned-Miller; Jan, Kautz (2018). Super slomo: High quality estimation of multiple intermediate frames for video interpolation. ICCV. arXiv:1712.00080.

[Shurui_Gui,_Chaoyue_Wang,_Qihua_Chen,_and_Dacheng_Tao-3] Shurui, Gui; Chaoyue, Wang; Qihua, Chen; Dacheng, Tao (2020). Featureflow: Robust video interpolation via structure-to-texture generation. IEEE. doi:10.1109/CVPR42600.2020.01402. ISBN 978-1-7281-7169-2.

[Myungsub_Choi,_Heewon_Kim,_Bohyung_Han,_Ning_Xu,_and_Kyoung_Mu_Lee-4] Myungsub, Choi; Heewon, Kim; Bohyung, Han; Ning, Xu; Kyoung, Mu Lee (2020). Channel attention is all you need for video frame interpolation. AAAI. doi:10.1609/aaai.v34i07.6693.

[1]

[2]

[3]

[4]