Introduction

A few months ago I started working on a project that could create a database of summarized YouTube videos using HuggingFace transformers and OpenAI GPT. I had put this project on pause because I couldn't find a good use case that I would actually use everyday. And due to recent life events, getting laid off, I can't see myself deep diving into this further. However, I wanted to share the project skeleton to whomeever might have some ideas/time.

Github Repo: https://github.com/freedomtowin/youtube-textract

Design Choices

I decided to create ONE container/service as opposed to having multiple microservices for each step in the pipeline. For multiple microservices I would need to use an orchestration tool, i.e., Step Functions, Airflow, etc, to chain the pipeline together, and I only had one use case for each step.

YouTube does has an API to extract video subtitles, however, it does not format the text with punctuations. Facebook’s Whisper transformer model, available on HuggingFace, seemed to provide a cleaner text output.

I had planned to share this project much earlier, but I didn’t have a clean way to share projects with sensitive secrets. Luckily, while learning front-end development, I found that some developers use two Docker compose and two Docker files to run local and cloud deployments.

The Pipeline

This project will use a variety of HuggingFace and OpenAI transformer models to 1) transcribe videos into text with Whisper 2) use sentence similarity transformers to separate text into paragraphs using all-MiniLM-L6-v2 as the embedding and 3) summarize and extract metadata using gpt-3.5-turbo and doctran.

At each step, the output of each model is saved into a table in an Athena database. I created an API Gateway\Lambda endpoint to query this database. The following code shows an example for how the endpoint can be called:

Example - The Home Depot - Instruction Extraction

Title: DIY Bed Frame: How to Make a Wooden Bed Frame | The Home Depot


OpenAI Output

The raw output from API Gateway for the video above, is shown below:

[{'channel_id': 'UCfB9yx0y0dUwQ0lpjH8R4gA',
                              'video_id': 'F9BOYvns08g',
                              'timestamp': '0',
                              'summary': 'This instructional video provides step-by-step instructions on how to make a standard wooden bed frame for a queen-size bed. The video suggests gathering the necessary materials such as wood screws, bed rail handles, and lumber, and measuring the area where the bed will be placed. It then explains how to cut the lumber, create pocket holes using a pocket hole jig, and assemble the frame crossmembers. The video also demonstrates how to attach the side apron, foot apron, head apron, legs, and center supports. Finally, it recommends sanding the frame, applying paint or stain, and provides a list of materials and tools needed for the project.',
                              'text_extract': '0. Gather necessary materials such as wood screws, bed rail handles, and lumber
                            1. Measure the area where the bed will be placed
                            2. Cut the lumber
                            3. Create pocket holes using a pocket hole jig
                            4. Assemble the frame crossmembers
                            5. Attach the side apron, foot apron, head apron, legs, and center supports
                            6. Sand the frame
                            7. Apply paint or stain
                            8. List of materials and tools needed for the project',
                              'text_category': 'step-process'}]

The formatted output is shown below (formatting done by ChatGPT):

Summary: This instructional video provides step-by-step instructions on how to make a standard wooden bed frame for a queen-size bed. The video suggests gathering the necessary materials such as wood screws, bed rail handles, and lumber, and measuring the area where the bed will be placed. It then explains how to cut the lumber, create pocket holes using a pocket hole jig, and assemble the frame crossmembers. The video also demonstrates how to attach the side apron, foot apron, head apron, legs, and center supports. Finally, it recommends sanding the frame, applying paint or stain, and provides a list of materials and tools needed for the project.

Steps:

  1. Gather necessary materials such as wood screws, bed rail handles, and lumber.

  2. Measure the area where the bed will be placed.

  3. Cut the lumber.

  4. Create pocket holes using a pocket hole jig.

  5. Assemble the frame crossmembers.

  6. Attach the side apron, foot apron, head apron, legs, and center supports.

  7. Sand the frame.

  8. Apply paint or stain.

  9. List of materials and tools needed for the project.

For completeness, the transcribed and processed YouTube audio is be shown below:

{'channel_id': 'UCfB9yx0y0dUwQ0lpjH8R4gA',
                              'video_id': 'F9BOYvns08g',
                              'timestamp': '0',
                              'transcript': "The bed is one of the main focal points of the bedroom. To ensure it catches your eye and the eyes of others, why not create your own? In this instructional video, you'll learn how to make a standard wooden bed frame,  for a queen-size bed. It complements our video on creating a wooden headboard. You'll need wood screws, bed rail handles, and other tools and materials that'll be covered a little later. Be sure to measure the area that the bed will occupy to confirm you have enough space. Account for accent furniture, doorway openings and space needed to maneuver around the bed. Once your measurements have been taken, gather the necessary amount of lumber to complete the project. Use wood glue for additional bonding support throughout the construction. To get started, cut the lumber pieces based on the dimensions outlined on your cut list,  provided at the end of this video. Then use a pocket hole jig to create pocket holes for your cut lumber pieces. The holes are drilled at an angle to hide the screws that connect the pieces together. The depth of your pocket holes are determined by the thickness of your cut lumber pieces. The Home Depot sells pocket hole jig kits that you can use to complete this step. Use the guides provided to adjust the thickness of the lumber you're working with. You can cover each hole with a pocket hole plug. Be sure to follow the complete instructions provided by your manufacturer when using any pocket hole kit. For this project, collect all of the frame crossmembers and drill two pocket holes on each end with the lumber pieces standing vertically. Lay out the frame crossmember so that you have one on each end of the frame sides. The middle crossmembers get placed at 13 1⁄2 inches and 27 inches on the frame. Once they are in place, screw them together. Repeat these steps to make the other half of the bed frame. Line up the long sides of the two frame assemblies. Clamp them together and make sure they're square. Mark the center of the conjoined assemblies at 18 inches. Then drill 3 1⁄8 inch holes to bolt the pieces together. Insert the 5 16ths by 4 inch bolt into the conjoined pieces with the accompanying cut washer, lock washer, and nut. Then tighten to secure it. Now may be a good time to transition the construction to a larger workspace if the area or surface you're utilizing isn't large enough."},
                             {'channel_id': 'UCfB9yx0y0dUwQ0lpjH8R4gA',
                              'video_id': 'F9BOYvns08g',
                              'timestamp': '160',
                              'transcript': "Prop the frame assembly by one and a half inches. You can use scrap pieces of lumber or the 2x4 and 1x4. Line one of the side apron pieces even with the ends of the frame assembly with a 1-1⁄2 inch overhang on the top edge. Clamp into place. Measure and drill holes for the bolts 3-1⁄4 inches down from the top edge, 1-1⁄2 inches on the overhang side. Do this at 8 inches, 29 inches, 52-1⁄2 inches, and 73 inches. The holes should be in the center of the 2x4 frame assembly. Repeat this step to attach the other side apron. Next, attach the foot apron. Line up the foot apron so that the ends are flush with the outer edge of the side apron pieces. Clamp and place. Drill holes for the bolt 3 and 1 quarter inches down from the top edge. Do this at 8 inches, 31 and 1 quarter inches, and 54 and a half inches. Insert and tighten the bolts in place. Repeat this step to attach the head apron. Flip over the frame assembly to attach the legs. One at a time, align them with the end of the side apron. Then secure in place with brad nails. In the same way that you did the side legs, you're going to attach the front and back legs. The only difference is that they will be aligned flush with the top of the apron and the face of the side legs. Next, attach the center supports onto the frame cross members. Place these center supports onto the cross frame members at 30 and a half inches. And these center supports at 24 inches onto the cross-frame members shown. Use your speed square to ensure the supports are square with the frame assembly. Secure in place with two and a half inch pocket hole screws. Finally, Finally, sand everything smooth, wipe it down to get rid of the sawdust, and apply a nice coat of paint or stain to complement your bedroom decor. Here are all the materials you'll need to build your wooden bed frame. 1 llwy de o laeth o ddŵr Here is a cut list for the lumber needed to complete this project. And lastly, you'll need the following tools for the job. For more detailed information about this and many other popular projects, check out the DIY Projects and Ideas section of thehomedepot.com or talk to one of our store associates. Good luck and thanks for shopping at The Home Depot."}

Conclusion

Me, knowing nothing about contructing wooden bed frames, have gotten some useful information and terminology about woodworking by using this simple tool. For example, I had no idea what "pocket holes using a pocket hole jig" means, but I could do some preemptive research on the topic before watch the video.

Now, is this useful? Maybe. There are many things you could potentially do with a database of YouTube transcripts, summaries, and OpenAI text processing. For example, you could create a custom search engine with a vectorized database such as LanceDB. If you are a creator, you could potentially crawl certain search terms to see if your work is mentioned anywhere (copywrite infringement).