視頻剪輯外文翻譯—高效視頻編輯移動應用程序

1、<p>　　字數：英文2331單詞，11235字符；中文3722漢字</p><p>　　出處：I Vegas,A Agrawal,T Tian.Efficient Video Editing for Mobile Applications[J]International Journal of Advanced Computer Science and Applications.2017,8(1):26

2、-30</p><p><b>　　外文文獻：　</b></p><p>　　Efficient Video Editing for Mobile Applications</p><p>　　Abstract Recording, storing and sharing video content has become one of the

3、 most popular usages of smartphones. This has resulted in demand for video editing apps that the users can use to edit their videos before sharing on various social networks. This study describes a technique to create a

4、video editing application that uses the processing power of both GPU and CPU to process various editing tasks. The results and subsequent discussion shows that using the processing power of both the GPU a</p><

5、p>　　Keywords—iOS programming; Image processing; GPU; CPU; Objective-C; GPUImage; OpenGL</p><p>　　I.INTRODUCTION</p><p>　　Smartphones have become an essential part of our day-to- day life. Ame

6、ricans spend about one hour a day on their smartphones using mobile applications. The iPhone is the most used device, occupying 47% of the smartphone market share.</p><p>　　We consume different types of cont

7、ent on our smartphones such as news, social-media, images, video games, music, films, TV shows, etc. Especially, the number of video content distributed around the Internet is growing exponentially every year due to popu

8、lar video hosting platforms like YouTube, Facebook, Snapchat and Instagram. The consumption of video in mobile platforms is expected to grow 67% year-on-year until 2019 as can be seen in Fig 1.</p><p>　　Fig.

9、 1. Evolution of Mobile video consumed</p><p>　　As a result of the high quality camera in iPhones, we can record video in high quality with a device that is always in our pocket. The videos can then be share

10、d with our friends across different social-media platforms. With more and more videos being recorded and shared, it has become important for the users to be able to edit those videos before being published on the Interne

11、t. Video editing is the process of manipulating video images, adding audio and/ or visual effects. Since smartphones are </p><p>　　This paper presents a study on developing a video editing application for iO

12、S platform. The application uses image processing algorithms and iOS programming techniques. Image processing is the processing of images using mathematical operations by using any form of signal processing for which the

13、 input is an image, a series of images, or a video and the output may be either an image or a set of characteristics or parameters related to the image. iOS programming techniques use a set of libraries, </p><

14、p>　　This application allows the user to record a video or to import a video stored in your iPhone camera roll. The user can select a specific part of the video and crop the video if it is required. The user can then a

15、dd some image filter effects along with a background song. Finally, the user can save the resulted video back to the iPhone.</p><p>　　II.METHODS</p><p>　　A.Technologies used</p><p>

16、　　The application is programmed in iOS version 9.0. iOS version 9.0 runs in 80% of the iOS devices using xCode version 7.3 and Objective-C as language development. Recently, Apple launched a new programming language for

17、iOS called Swift. This application however is programmed in Objective-C instead of Swift since Objective-C is a more evolved language with more documentation about video processing than Swift.</p><p>　　B.Lib

18、raries used</p><p>　　For the entire iOS application flow and user interface, we have used the Cocoa Framework, a group of native libraries provided by Apple to create the user interface of an application.<

19、;/p><p>　　The video capture, video importing/exporting and video cropping, is implemented using UIImagePickerController. This is a class created by Apple to work with media files.</p><p>　　The vide

20、o filter processing is created using GPUImage, a third-party library created by Brad Larson. This library gives you the opportunity to use the GPU to process the video instead of CPU. The video processing tools provided

21、by Apple only allows to process video using CPU. Also, using GPUImage you can use predefined filters or you can create filters of your own.</p><p>　　To preview the video, the application uses Core Image, an

22、iOS native library that allows you to reproduce media content in your application.</p><p>　　AVFoundation is used to add custom background audio to the videos. This is a native iOS library provided by Apple t

23、o manipulate audio in media files.</p><p><b>　　C.Views</b></p><p>　　In iOS, when we talk about a view, we are referring to a screen in the application. Our application has four diffe

24、rent views, as discussed below.</p><p>　　The first view allows the user to select a video for editing. The user can select between recording a video using the iphone camera and importing a video from the iPh

25、one camera roll. The user can also select certain parts of the video to be processed, and delete the rest of the video. The new video segment, thus created, is saved in a temporary directory inside the application.</p

26、><p>　　Once the video is selected for editing, the filter screen appears. This view provides a preview of the video where the user can select a filter to apply. There is an option to keep the video as it is wit

27、hout applying any filters. When a filter is selected, the application sends the video to the GPU. This means that the CPU is not processing the video, as the GPU works as a separate thread. While the video is being proce

28、ssed, a loading icon is displayed. When the process is complete, the processe</p><p>　　The third view is the audio view. This view shows a classical iOS TableView with a list of all the available songs that

29、can be chosen for the video. The song files are stored with the application, as the application only offers a few songs and the durations are not longer than twenty seconds. When the user selects a song, the video is pro

30、cessed again. The processing uses the CPU by creating a parallel thread, so now the application continues to run in the main thread. The user also has the optio</p><p>　　The fourth view offers a final previe

31、w of the video with the new audio included. Here, the user has the option to save the video to the camera roll. Note that, so far, the video is only stored in a temporary folder. This is being done to prevent unnecessary

32、 use of memory space and CPU as it is more efficient to work with a file stored in a temporary directory inside the application space.</p><p><b>　　D.Filters</b></p><p>　　GPUImage wor

33、ks on top of OpenGL shaders. OpenGL Shaders are programs designed to run on some stage of a graphic processor (GPU). As a result, our application can process videos using GPU and also use predefined image filters or crea

34、te a custom filter using OpenGL features.</p><p>　　As mentioned earlier, when the application starts processing the video, the CPU creates a parallel thread. This parallel thread is then processed by the GPU

35、 as shown in Fig.2.The GPU reads every frame of the video and processes each frame separately. When all the frames are processed, the GPU returns the control back to the CPU.</p><p>　　Fig. 2. GPU and CPU sta

36、te while processing a video</p><p>　　The process that OpenGL Shaders use to process an image is called rendering pipeline. The OpenGL rendering pipeline defines a number of stages for this shaders as shown i

37、n Fig. 3.</p><p>　　Fig. 3. States of the rendering pipeline</p><p>　　The vertex shader transforms the video images into individual vertices. Primitive assembly connects the vertices created by t

38、he vertex shader to form figures which are called primitive units. Rasterization is then applied, which transforms the primitive units into smaller units called fragments. In the fragment processing stage, colors and tex

39、tures are applied to the fragments, which is then saved in a Frame Buffer. The frame buffer allows us to create an image or show the image on a screen. The</p><p>　　III.RESULTS</p><p>　　Fig. 4 d

40、isplays the first view of the application. In this view, you can select two options; Record a video using the iPhone camera or import a video from the camera roll. Fig. 5 shows the view where the user can crop the video.

41、 Fig. 6 shows the view where a filter can be applied to the video. The application currently provides 15 popular filters, as shown in Table I.</p><p>　　Fig. 4. First app view with two available options</p

42、><p>　　Fig. 5. Second app view to crop the video</p><p>　　Fig. 6. Third app view to apply filters</p><p>　　TABLE I.FILTERS AVAILABLE</p><p>　　Fig. 7 provides a view where

43、 the user can choose an audio song that will be added to the video. Currently, the application provides 10 audio songs. These songs have been downloaded from jammendo.com, which are under Creative Commons license. The la

44、st view, as shown in Fig. 8, provides a preview of the processed video and gives user the option to save the video on the camera roll.</p><p>　　Fig. 7. Fourth app view to select an audio song</p><

45、p>　　Fig. 8. Fifth app view to save the edited video</p><p>　　IV.DISCUSSION</p><p>　　A.Using GPU or CPU for image processing</p><p>　　In the methods section, we mentioned about us

46、ing GPU processing alongside the CPU processing for many of the processing tasks. For parallel operations like video processing, using GPU has significant performance advantages over CPU. The GPUImage framework takes onl

47、y 2.5 ms on an iPhone 4 to upload a frame from the camera, apply a gamma filter, and display, versus 106 ms for the same operation using Core Image and 460 ms using CPU-based processing. This makes GPUImage 40X faster th

48、an Core Image a</p><p>　　CoreImage is the library provided by Apple to process images and video files. In newer devices like the iPhone 6, we can achieve the same performance using CPU or GPU. However, for t

49、his study we decided to use GPU processing because older devices like the iPhone 4 and iPhone 5 are more responsive when we utilize both GPU and CPU for video editing tasks.</p><p>　　B.Duration of the videos

50、</p><p>　　Several social networks such as Instagram and Snapchat limit the length of videos that can be uploaded to 10 or 15 seconds. When a user uses a mobile application, they want a fast, responsive and a

51、 seamless user experience, and processing a video longer than 20 seconds can take a longer time thus negatively impacting the user experience. So, we decided to limit the duration of the videos that the users can take us

52、ing the application to 20 seconds. Table II shows the video processing time using t</p><p>　　TABLE II. 1080P VIDEO PROCESSING TIME ON AN IPHONE 6</p><p>　　Table III shows the video processing ti

53、me using the application for videos of different durations in 640p (unlike videos in Table II that are in 1080p). </p><p>　　TABLE III. 640P VIDEO PROCESSING TIME ON AN IPHONE 6</p><p>　　Table IV

54、 shows the video processing to apply blur effect using the application on iPhone 5s. All videos are in 1080p with 30 frames per second. As the data shows, the processing time required by an iPhone 5s is almost the double

55、 of the time required by iPhone 6 as shown in Table II. This is as a result of the iPhone 5s GPU being half as powerful as the iPhone 6 GPU.</p><p>　　TABLE IV. 1080P VIDEO PROCESSING TIME ON AN IPHONE 5S<

56、/p><p>　　C.Future Work</p><p>　　Future work will involve improving the scalability of the application. For instance, we will have the songs list stored on the server. The application will be able t

57、o connect to the server and the songs can be downloaded to the smartphone. Another new feature will involve adding the option to select between different image qualities for the output video. With lower quality export vi

58、deos, the processing will be faster as compared to a video generated using the high quality option.</p><p>　　V.CONCLUSION</p><p>　　Mobile applications and video content have become an integral p

59、art of our lives. It has become common for people to use their mobile phones to record, edit and share videos on social networking sites. This paper presents a video editing application for iOS devices that can be used t

60、o record videos and edit them. The edit features include cropping, applying filters or adding background audio. The application describes a technique to use the processing power of both the GPU and the CPU to improve <

61、;/p><p><b>　　中文譯文：</b></p><p>　　高效視頻編輯移動應用程序</p><p>　　摘要 錄制、存儲和共享視頻內容已成為智能手機最受歡迎的應用之一。這導致了對視頻編輯應用程序的需求，用戶可以使用它來編輯視頻，然后再發(fā)布到社交網絡上。本研究介紹了一種創(chuàng)建視頻編輯應用程序的技術，該應用程序使用GPU和CPU的處理能力來處

62、理各種編輯任務。結果和后續(xù)討論表明，與僅基于CPU的處理相比，在視頻編輯過程中使用GPU和CPU的處理能力會使得應用程序的時間效率和響應速度更快。</p><p>　　關鍵詞——iOS編程；圖像處理；GPU；CPU；Objective-C；GPUImage；OpenGL</p><p><b>　　1.簡介</b></p><p>　　智能手機

63、已成為我們日常生活的重要組成部分。美國人每天大約花一個小時在他們的智能手機上使用移動應用程序。iPhone是最常用的設備，占智能手機市場份額的47％。</p><p>　　我們在智能手機上使用不同類型的內容，如新聞、社交媒體、圖像、視頻游戲、音樂、電影、電視節(jié)目等。特別是，由于受歡迎的視頻托管平臺，如YouTube、Facebook、Snapchat和Instagram，互聯網上發(fā)布的視頻內容數量每年都在成倍增長

64、。預計到2019年，移動平臺的視頻消費量將同比增長67％，如圖1所示。</p><p>　　圖1.移動視頻消費的發(fā)展</p><p>　　iPhone具有高品質的攝像頭，我們可以用其設備錄制高質量的視頻。這些視頻可以通過不同的社交媒體平臺與我們的朋友分享。隨著越來越多的視頻被錄制和共享，用戶在發(fā)布互聯網之前就能夠編輯這些視頻變得非常重要。視頻編輯是處理視頻圖像、添加音頻和/或視覺效果的過程

65、。由于智能手機在處理和內存方面越來越強大，因此可以構建iPhone應用程序來編輯用戶錄制的視頻，而無需計算機，并且具有更好更快的用戶體驗。</p><p>　　本文介紹了一個開發(fā)iOS平臺視頻編輯應用程序的研究。該應用程序使用圖像處理算法和iOS編程技術。圖像處理是使用數學運算處理圖像的過程，它使用任何形式的信號處理，其中輸入是圖像、一系列圖像或視頻，輸出可以是圖像或與圖像相關的一組特征或參數。iOS編程技術使用

66、了一組用于構建iPhone應用程序的庫、算法和最佳實踐。</p><p>　　此應用程序允許用戶錄制視頻或導入存儲在iPhone相機相冊中的視頻。用戶可以選擇視頻的特定部分，并在需要時裁剪視頻。用戶可以添加一些圖像濾鏡效果以及背景歌曲。最后，用戶可以將結果視頻保存回iPhone。</p><p><b>　　2.方法</b></p><p>&

67、lt;b>　　A.技術應用</b></p><p>　　該應用程序在iOS版本9.0中進行編程。iOS版本9.0運行在80%的iOS設備中，使用xCode7.3和Objective-C作為語言開發(fā)。最近，Apple推出了一款名為Swift的iOS新編程語言。然而，這個應用程序是在Objective-C中編寫的，而不是Swift，因為Objective-C是一種更進化的語言，其中有關于視頻處理的文

68、檔比Swift更多。</p><p><b>　　B.庫應用</b></p><p>　　對于整個iOS應用程序流和用戶界面，我們使用了Cocoa Framework，這是Apple提供的一組本機庫，用于創(chuàng)建應用程序的用戶界面。</p><p>　　視頻采集、視頻導入/導出和視頻裁剪使用UIImagePickerController來實現。這是

69、Apple創(chuàng)建的用于處理媒體文件的一個分類。</p><p>　　視頻濾鏡處理是使用由BradLarson創(chuàng)建的第三方庫GPUImage來實現的。該庫使用戶有機會使用GPU來處理視頻而不是CPU。Apple提供的視頻處理工具只允許使用CPU處理視頻。另外，使用GPUImage可以使用預定義的濾鏡，也可以創(chuàng)建自己的濾鏡。</p><p>　　要預覽視頻，應用程序使用了Core Image，這

70、是一個iOS本機庫，允許用戶在應用程序中重現媒體內容。</p><p>　　AVFoundation用于為視頻添加自定義背景音頻。這是Apple提供的的本機iOS庫，用于處理媒體文件中的音頻。</p><p><b>　　C.視圖</b></p><p>　　在iOS中，當我們談論一個視圖時，我們指的是應用程序中的一個屏幕。我們的應用程序有四種

71、不同的視圖，如下所述。</p><p>　　第一個視圖允許用戶選擇一個視頻進行編輯。用戶可以選擇使用iPhone相機錄制視頻和從iPhone相機相冊導入視頻。用戶還可以選擇要處理的視頻的某些部分，并刪除視頻的其余部分。新創(chuàng)建的視頻片段保存在應用程序內的臨時目錄中。</p><p>　　一旦選擇了視頻進行編輯，就會出現濾鏡屏幕。此視圖提供視頻預覽，用戶可以在其中選擇要應用的濾鏡。有一個選項可

72、以保留視頻，因為它不需要應用任何濾鏡。當選擇濾鏡時，應用程序將視頻發(fā)送給GPU。這意味著CPU不處理視頻，因為GPU可以作為單獨的線程工作。在處理視頻時，會顯示一個加載圖標。該過程完成后，可以使用應用的濾鏡查看已處理的視頻。如果用戶不喜歡應用的濾鏡，他們可以選擇另一個濾鏡并重復上述過程。視頻處理完畢后，它仍保留在臨時目錄中。</p><p>　　第三個視圖是音頻視圖。該視圖顯示了一個經典的iOSTableView

73、，其中列出了可供視頻選擇的所有可用歌曲。歌曲文件與應用程序一起存儲，因為應用程序僅提供幾首歌曲，并且持續(xù)時間不超過20秒。當用戶選擇一首歌曲時，視頻會再次被處理。該處理使用CPU創(chuàng)建一個并行線程，所以現在應用程序繼續(xù)在主線程中運行。用戶還可以選擇在視頻中不添加任何歌曲。音頻歌曲添加到視頻后，視頻再次保存在臨時目錄中。</p><p>　　第四個視圖提供了包含新音頻的視頻的最終預覽。在這里，用戶可以選擇將視頻保存到

74、相機相冊。請注意，到目前為止，視頻只存儲在臨時文件夾中。這是為了防止不必要地使用內存空間和CPU，因為使用存儲在應用程序空間內的臨時目錄中的文件更有效。</p><p><b>　　D.濾鏡</b></p><p>　　GPUImage在OpenGL著色器的基礎上工作。OpenGL著色器是設計用于在圖形處理器（GPU）的某個階段上運行的程序。因此，我們的應用程序可以使

75、用GPU處理視頻，并使用預定義的圖像濾鏡或使用OpenGL功能創(chuàng)建自定義濾鏡。</p><p>　　如前所述，當應用程序開始處理視頻時，CPU會創(chuàng)建一個并行線程。然后這個并行線程由GPU處理，如圖2所示。GPU讀取視頻的每一幀并分別處理每一幀。GPU處理完所有幀后，GPU將控制權返還給CPU。</p><p>　　圖2.處理視頻時GPU和CPU的狀態(tài)</p><p>

76、;　　OpenGL著色器用于處理圖像的過程稱為渲染管線。OpenGL渲染管線為這個著色器定義了多個階段，如圖3所示。</p><p>　　圖3.渲染管線的狀態(tài)</p><p>　　頂點著色器將視頻圖像轉換為單個頂點。原始程序集連接由頂點著色器創(chuàng)建的頂點以形成被稱為基本單元的圖形。然后應用光柵化，將基元單位轉換為稱為片段的較小單位。在片段處理階段，將顏色和紋理應用于片段，然后將其保存在幀緩沖

77、區(qū)中。幀緩沖區(qū)允許我們在屏幕上創(chuàng)建圖像或顯示圖像。使用OpenGL著色器的關鍵優(yōu)勢在于可以在GPU中并行運行各種操作，從而實現更加靈敏的應用程序。</p><p><b>　　3.結果</b></p><p>　　圖4顯示了應用程序的第一個視圖。在此視圖中，用戶可以選擇兩個選項；使用iPhone相機錄制視頻或從相機相冊導入視頻。圖5顯示了用戶可以裁剪視頻的視圖。圖6顯

78、示了濾鏡可以應用于視頻的視圖。該應用程序目前提供了15個流行的濾鏡，如表1所示。</p><p>　　圖4.具有兩個可用選項的第一個應用程序視圖</p><p>　　圖5.用于裁剪視頻的第二個應用程序視圖</p><p>　　圖6.應用濾鏡的第三個應用程序視圖</p><p><b>　　表1.提供的濾鏡</b><

79、/p><p>　　圖7提供了一個視圖，用戶可以在其中選擇將被添加到視頻的音頻歌曲。目前，該應用程序提供了10首音頻歌曲。這些歌曲已從jammendo.com下載，它們受Creative Commons的許可。最后一個視圖，如圖8所示，提供了經過處理的視頻的預覽，并給用戶選擇將視頻保存在相機相冊中。</p><p>　　圖7.選擇音頻歌曲的第四個應用程序視圖</p><p&g

80、t;　　圖8.保存編輯視頻的第五個應用程序視圖</p><p><b>　　四.討論</b></p><p>　　A.使用GPU或CPU進行圖像處理</p><p>　　在“方法”部分，我們提到了一起使用GPU喝CPU來處理許多任務。對于像視頻處理這樣的并行操作，使用GPU比CPU具有顯著的性能優(yōu)勢。在iPhone4上，GPUImage框架只需

81、要2.5毫秒即可從相機上傳幀，應用伽馬濾鏡并顯示，而使用CoreImage執(zhí)行相同操作的時間為106ms，使用基于CPU的處理時為460ms。這使得GPUImage比CoreImage快40倍，比基于CPU的處理快184倍。在iPhone4S上，對于這種情況，GPUImage比CoreImage快4倍，比基于CPU的處理快了102倍。</p><p>　　CoreImage是Apple提供的用于處理圖像和視頻文件

82、的庫。在像iPhone6這樣的新型設備中，我們可以使用CPU或GPU實現相同的性能。但是，對于這項研究，我們決定使用GPU處理，因為當我們同時使用GPU和CPU進行視頻編輯任務時，像iPhone4和iPhone5這樣的老設備會更加靈敏。</p><p><b>　　B.視頻的持續(xù)時間</b></p><p>　　Instagram和Snapchat等幾個社交網絡將可上

83、傳視頻的時間長度限制在10秒或15秒。當用戶使用移動應用程序時，他們需要快速、靈敏且無縫的用戶體驗，處理超過20秒的視頻可能需要較長的時間，從而對用戶體驗產生負面影響。因此，我們決定將用戶可以使用該應用程序的視頻的持續(xù)時間限制為20秒。表2顯示了使用具有不同視頻持續(xù)時間的應用程序的視頻處理時間。所有視頻都是1080p，每秒30幀。對于這個實驗，使用iPhone6應用了模糊效果。</p><p>　　表2.iPho

84、ne 6上的1080p視頻處理時間</p><p>　　表3顯示了在640p中使用不同持續(xù)時間的視頻應用程序的視頻處理時間（與表2中1080p視頻不同）。</p><p>　　表3.iPhone 6上的640p視頻處理時間</p><p>　　表4顯示了使用iPhone5s上的應用程序來應用模糊效果的視頻處理。所有視頻都是1080p，每秒30幀。如數據所示，iPho

85、ne5所需的處理時間幾乎是表2中iPhone6所需時間的兩倍。這是由于iPhone5s GPU的功能僅為iPhone6 GPU的一半。</p><p>　　表4.iPhone 5s上的1080p視頻處理時間</p><p><b>　　C.未來的工作</b></p><p>　　未來的工作將涉及提高應用程序的可伸縮性。例如，我們將在服務器上存儲

86、歌曲列表。該應用程序將能夠連接到服務器，歌曲可以下載到智能手機。另一項新功能將包括添加選項，以在不同圖像質量的輸出視頻之間進行選擇。使用較低質量的導出視頻，與使用高質量選項生成的視頻相比，處理速度會更快。</p><p><b>　　5.結論</b></p><p>　　移動應用程序和視頻內容已成為我們生活中不可分割的一部分。人們經常使用手機在社交網站上錄制、編輯和分