CyberCode Academy

CyberCode Academy
  • 0.0 (0)
  • Courses
  • Updated Daily

Welcome to CyberCode Academy — your audio classroom for Programming and Cybersecurity. 🎧 Each course is divided into a series of short, focused episodes that take you from beginner to advanced level — one lesson at a time. From Python and web development to ethical hacking and digital defense, our content transforms complex concepts into simple, engaging audio learning. Study anywhere, anytime — and level up your skills with CyberCode Academy. 🚀 Learn. Code. Secure. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

  1. 5h ago

    Course 40 - Web Scraping with Python | Episode 17: Mastering Requests, Regex, and Beautiful Soup

    In this lesson, you’ll learn about: how Python retrieves web pages, how regex is used for pattern-based extraction, and how BeautifulSoup improves scraping by understanding HTML structure instead of treating it as plain text1. Fetching Web Content in Python🔹 HTTP Request FlowWeb scraping always starts with getting the page content.🔹 Libraries Used urllib → built-in, basic controlhttplib2 → low-level controlrequests → easiest and most popular🔹 Requests Exampleimport requests response = requests.get("https://example.com") html = response.text 🔹 User-Agent HandlingSome sites block bots, so you can:headers = {"User-Agent": "Mozilla/5.0"} requests.get(url, headers=headers) 👉 Key Insight Without proper headers, many sites will reject your scraper2. Regular Expressions (Regex Basics)🔹 Pattern Matching ConceptRegex treats web data as raw text patterns.3. Core Regex FunctionsFunctionBehaviormatch()checks start onlysearch()finds first match anywherefindall()returns all matches🔹 Special SymbolsSymbolMeaning\ddigits\wletters + numbers\swhitespace🔹 Example Patternimport re re.findall(r"\d+", "Price is 123 dollars") 👉 Key Insight Regex is powerful but fragile for HTML4. Advanced Regex Techniques🔹 Ranges & Groups [A-Z] → uppercase letters{3} → exact repetition( ) → capture groups🔹 Example: Extract Namesre.search(r"(\w+) (\w+)", "John Smith") 5. Real Web Scraping Use Cases🔹 Inspecting HTMLUsing browser tools, you can locate: items, headerscontact detailslocation data🔹 Example Targets Phone numbersZip codesCity/state data6. BeautifulSoup (Structured Parsing)🔹 DOM-Based ApproachBeautifulSoup understands HTML as a tree structure, not text.🔹 Basic Usagefrom bs4 import BeautifulSoup soup = BeautifulSoup(html, "lxml") print(soup.title.string) 🔹 Key Advantage Navigates tags easilyHandles broken HTMLCleaner extraction than regex7. Parsers (LXML vs HTML5lib)ParserStrengthlxmlfasthtml5libvery forgiving👉 Key Insight Parser choice affects speed vs accuracy8. Regex vs BeautifulSoupFeatureRegexBeautifulSoupStructure aware❌✔️Speed✔️MediumReliability❌✔️9. Mental ModelThink of scraping like: 📥 Requests → download page🔍 Regex → pattern hunting🌳 BeautifulSoup → structured navigationFinal TakeawayWeb scraping becomes powerful when you stop treating HTML as text and start treating it as a structured tree of data.👉 Use: Requests → fetchRegex → quick patternsBeautifulSoup → real extractionThat combination covers most real-world scraping tasks. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 17: Mastering Requests, Regex, and Beautiful Soup

  2. 1d ago

    Course 40 - Web Scraping with Python | Episode 16: Mastering Data Extraction with Beautiful Soup

    In this lesson, you’ll learn about: how web scraping works end-to-end, why fetching and parsing are the two core stages, and how different tools like Regex, BeautifulSoup, and Scrapy compare in real-world data extraction1. What is Web Scraping?🔹 Core IdeaWeb scraping = automated data extraction from websitesInstead of manually copying data, a program: Visits a pageReads the HTMLExtracts structured information2. Two-Phase Scraping Workflow🔹 Overall PipelinePhase 1: Fetching Content Send HTTP request (GET)Receive HTML responseStore raw page contentTools: Requestsurllibhttplib2Phase 2: Parsing & Extraction Analyze HTML structureExtract required dataClean results3. Regex vs Structured Parsers🔹 Regular ExpressionsRegex: Works on text patternsFast but fragileBreaks easily on messy HTML👉 Key Insight HTML is not flat text—it’s structured data4. BeautifulSoup (Structure-Aware Parsing)🔹 Why It Works BetterBeautifulSoup: Understands HTML tree structureFixes broken markupLets you navigate elements easily🔹 Key AdvantageInstead of guessing text patterns:👉 you navigate the DOM like a tree5. HTML vs DOM ParsingTypeDescriptionHTML parsingRaw server outputDOM parsingRendered browser structure🔹 Important Difference HTML = static snapshotDOM = live, updated by JavaScript6. Static vs Dynamic Content🔹 Static Pages Easy to scrapeNo JavaScript requiredBeautifulSoup works well🔹 Dynamic Pages Content generated by JavaScriptRequires browser renderingTools: SeleniumScrapyHeadless browsers👉 Key Insight If data appears after page load → you need a browser engine7. Advanced Tools Overview🔹 Scrapy (Industrial Tool) Built for scaleHandles crawling + pipelinesUsed for production systems🔹 Selenium Controls real browserHandles JavaScriptSlower but powerful🔹 Computer Vision Scraping (Sikuli) Reads screen pixelsWorks without HTMLUsed when UI has no accessible structure8. Mental ModelThink of scraping as: 📥 Fetch → download the page🧠 Parse → understand structure🎯 Extract → get useful dataFinal TakeawayWeb scraping is not just “copying data”—it’s a structured pipeline:👉 fetch → parse → extract → transformAnd the tool you choose depends on one question:Is the data static HTML or dynamically generated?That single decision determines everything else. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 16: Mastering Data Extraction with Beautiful Soup

  3. 2d ago

    Course 40 - Web Scraping with Python | Episode 15: Mastering Items, Loaders, and Processing Pipelines

    In this lesson, you’ll learn about: how Scrapy structures scraped data using Items, how Item Loaders simplify extraction and cleaning, and how Pipelines transform raw scraped output into usable datasets1. Scrapy Items (Structured Data Containers)🔹 What Are Items?Scrapy Items are structured containers for scraped data.Think of them as:a strongly-typed dictionary for scraped content🔹 Example Structureclass StockItem(scrapy.Item): name = scrapy.Field() symbol = scrapy.Field() price = scrapy.Field() 👉 Key Insight Items force structure into messy web data2. Using Items in Scrapy Shell🔹 Manual Assignment FlowYou can: Test XPath selectorsExtract values manuallyAssign them into Items🔹 Exampleitem["name"] = response.xpath("//h1/text()").get() item["price"] = response.xpath("//fin-streamer/text()").get() 👉 Key Insight Scrapy Shell helps you validate structure before automation3. Project-Based Item Integration🔹 Moving into Real SpidersItems are defined in:items.py Then used inside spiders:yield StockItem( name=name, symbol=symbol, price=price ) 👉 Key Insight Items enforce consistency across your whole scraping system4. Exporting Data (CSV / JSON)🔹 Built-in Export Systemscrapy crawl stocks -o data.csv 🔹 Output Formats CSV → analyticsJSON → APIsXML → legacy systems👉 Key Insight Scrapy can export structured data without extra libraries5. Item Loaders (Automation Layer)🔹 Why They ExistItem Loaders reduce repetitive code and handle transformation automatically.🔹 Example Usageloader.add_xpath("price", "//span/text()") 6. Input & Output Processors🔹 MapCompose (Input Cleaning)from scrapy.loader.processors import MapCompose Used to: Clean URLsFormat stringsConvert data types🔹 TakeFirst (Output Simplification)from scrapy.loader.processors import TakeFirst Used to: Convert lists → single values👉 Key Insight Processors turn raw extraction into clean structured data automatically7. Pipelines (Post-Processing System)🔹 What Happens After ScrapingPipelines run after data extraction🔹 Example Pipelineclass PriceFilterPipeline: def process_item(self, item, spider): if float(item["price"]) > 100: item["high_value"] = True return item 👉 Key Insight Pipelines are where business logic lives8. Enabling PipelinesIn settings.py:ITEM_PIPELINES = { "myproject.pipelines.PriceFilterPipeline": 300, } Lower number = higher priority9. Full Data Flow Model Spider extracts dataItems structure itItem Loaders clean itPipelines transform itExport stores it10. Mental ModelThink of Scrapy like a factory: 🕷️ Spider → collector📦 Items → containers🧼 Loaders → cleaning station🏭 Pipelines → production lineFinal TakeawayScrapy is not just about scraping—it’s about turning raw web data into structured, validated datasets automatically.Once you master Items → Loaders → Pipelines:👉 you stop “extracting data” 👉 and start engineering data systems You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 15: Mastering Items, Loaders, and Processing Pipelines

  4. 3d ago

    Course 40 - Web Scraping with Python | Episode 14: Building and Automating Custom Spiders with the Scrapy Framework

    In this lesson, you’ll learn about: Scrapy’s full architecture, how to build real spiders from scratch, and how to move from simple extraction to production-ready crawling with structured data pipelines1. Scrapy Architecture (How Everything Works)🔹 Core System FlowScrapy is built around a central engine that coordinates everything.🔹 Main ComponentsComponentRoleEngineControls flowSchedulerQueues URLsDownloaderFetches pagesSpiderExtracts dataPipelineProcesses & stores data👉 Key Insight You don’t control HTTP manually—Scrapy does it for you2. Project Setup & Spider Creation🔹 Initialize a Projectscrapy startproject myproject 🔹 Generate a Spiderscrapy genspider stocks yahoo.com 🔹 Project Structuremyproject/ ├── spiders/ ├── items.py ├── pipelines.py ├── settings.py 👉 Key Insight Each file has a strict responsibility → clean separation of logic3. Extracting Real Data (Yahoo Finance Example)🔹 Target Use CaseWe extract: Company nameStock priceMarket data🔹 XPath in Spiderdef parse(self, response): yield { "name": response.xpath("//h1/text()").get(), "price": response.xpath("//fin-streamer[@data-field='regularMarketPrice']/text()").get() } 👉 Key Insight Spiders are just Python classes with extraction rules4. Running the Spider🔹 Execution Commandscrapy crawl stocks 🔹 Output Options Console printJSON exportCSV exportFile writing🔹 Save to Filescrapy crawl stocks -o data.json 👉 Key Insight Scrapy supports structured output without extra code5. Item Loaders (Cleaner Code)🔹 Why They MatterItem Loaders help: Clean dataNormalize valuesReduce repeated logic🔹 Examplefrom scrapy.loader import ItemLoader loader = ItemLoader(item=StockItem(), response=response) loader.add_xpath("price", "//span/text()") return loader.load_item() 👉 Key Insight You separate extraction from transformation6. Pipelines (Final Processing Layer)🔹 What Pipelines Do Clean dataValidate dataSave to database/files🔹 Example Pipelineclass CleanPipeline: def process_item(self, item, spider): item["price"] = float(item["price"]) return item 👉 Key Insight Pipelines act like a data factory assembly line7. Full Data Flow Scheduler queues URLDownloader fetches pageSpider extracts dataPipeline cleans itOutput stored8. Mental ModelThink of Scrapy as: 🧠 Brain → Engine📦 Factory line → Pipelines🕷️ Workers → Spiders🚚 Delivery system → DownloaderFinal TakeawayScrapy turns scraping into a fully automated data engineering system.Once you combine: Spiders (logic)Selectors (extraction)Pipelines (processing)👉 You don’t just collect data anymore—you build production-grade data pipelines. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 14: Building and Automating Custom Spiders with the Scrapy Framework

  5. 4d ago

    Course 40 - Web Scraping with Python | Episode 13: Mastering Scrapy Shell, CSS, and XPath Selectors

    In this lesson, you’ll learn about: how to use Scrapy Shell for interactive crawling, how CSS selectors work for fast extraction, and how XPath enables advanced and flexible data targeting1. What is Scrapy Shell?🔹 Interactive Prototyping ToolScrapy Shell is a live testing environment where you can: Test selectors before writing spidersInspect HTML responses instantlyExperiment with scraping logic🔹 Key Objects Inside Shell response → HTML content of the pagerequest → HTTP request detailsspider → scraper context👉 Key Insight You can test everything before writing real crawling logic2. Working with Live URLs and FilesScrapy Shell supports: 🌐 Live websites📄 Local HTML files👉 This makes it ideal for debugging broken or complex pages3. CSS Selectors (Fast & Simple)🔹 Basic Extraction🔹 Common SyntaxSelectorMeaning#idSelect by ID.classSelect by classtagSelect by tag🔹 Scrapy Shell Methodsresponse.css("title").get() response.css("p").get_all() 🔹 Extract Attributesresponse.css("img::attr(src)").get() 👉 Key Insight CSS is perfect for quick, readable extraction4. Important Behavior: Cached Responses🔹 One Hidden DetailScrapy Shell: Works on cached HTMLWon’t reflect live changes unless restarted👉 Key Insight Always restart shell when debugging updated pages5. XPath Selectors (Advanced Power)🔹 Full DOM NavigationXPath lets you navigate HTML like a tree structure6. Absolute vs Relative XPath🔹 Absolute Path/html/body/div/p Starts from rootVery strict🔹 Relative Path//div/p Searches anywhereMore flexible7. Attribute Matching in XPath🔹 Using @response.xpath("//img[@id='logo']").get() 🔹 Using Wildcardsresponse.xpath("//*[@id='main']").get() 8. Using contains()🔹 Pattern Matchingresponse.xpath("//p[contains(text(), 'news')]").get() 👉 Key Insight XPath is powerful for uncertain or messy HTML structures9. CSS vs XPathFeatureCSSXPathSimplicity✔️ Easy❌ More complexPower❌ Limited✔️ Very powerfulFlexibilityMediumVery high10. Mental ModelThink of Scrapy Shell as: A laboratoryCSS = quick filtersXPath = surgical precision toolsFinal TakeawayScrapy Shell bridges the gap between:👉 “guessing selectors” and 👉 “engineered extraction logic”Once you master CSS + XPath inside the shell, you can confidently build spiders that work on even the most complex websites. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 13: Mastering Scrapy Shell, CSS, and XPath Selectors

  6. 5d ago

    Course 40 - Web Scraping with Python | Episode 12: From Parsing Foundations to Scrapy Essentials

    In this lesson, you’ll learn about: how Scrapy turns simple scraping into large-scale crawling systems, the difference between scraping and crawling, and how to use a framework-driven approach for industrial web data extraction1. From Parsing to Real-World Crawling🔹 HTML vs DOM Parsing🔹 Key DifferenceTypeWhat it seesHTML parsingRaw server responseDOM parsingFinal rendered page👉 Key Insight JavaScript can completely change what your scraper sees after load2. Scraping vs Crawling🔹 Two Levels of Data CollectionConceptScopeScrapingSpecific pages/dataCrawlingEntire websites🔹 Real-World Analogy Scraping → reading one articleCrawling → reading the entire library3. Why Scrapy Exists🔹 The Framework AdvantageScrapy is not just a tool—it is a framework.👉 It controls execution and calls your code🔹 Inversion of ControlInstead of:you controlling everythingScrapy:controls the flow and executes your logic4. Core Scrapy Concepts🔹 Spider System Defines what to crawlDefines how to parse dataSends requests automatically🔹 Engine Flow Scheduler queues URLsEngine sends requestsSpider processes responsesPipeline stores data5. Getting Started Tools🔹 Installationpip install scrapy 🔹 Useful Commands scrapy bench → performance testscrapy fetch URL → download raw HTMLscrapy view URL → see rendered page👉 Key Insight These tools let you inspect how Scrapy “sees” the web6. Scrapy Shell (Prototyping Tool)🔹 Interactive TestingUse it to: Test selectorsDebug parsing logicInspect live responses7. CSS vs XPath Selectors🔹 Two Ways to Target DataMethodStrengthCSSSimple & readableXPathPowerful & flexible🔹 Exampleresponse.css("div.title").get() response.xpath("//div[@class='title']").get() 👉 Key Insight XPath can navigate complex structures CSS cannot8. Performance Thinking🔹 Why Scrapy is Fast Asynchronous requestsBuilt-in schedulerEfficient pipelines🔹 Metrics You Monitor Pages per minuteResponse sizeCrawl depth9. Mental ModelThink of Scrapy as: A robot armyA data factoryA controlled pipeline systemYou only define: 👉 what to collect 👉 how to parse itFinal TakeawayScrapy is where web scraping becomes engineering instead of scripting.Once you understand its structure: Scraping becomes scalableCrawling becomes automatedData collection becomes production-gradeAnd you stop writing scripts… and start building systems. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 12: From Parsing Foundations to Scrapy Essentials

  7. 6d ago

    Course 40 - Web Scraping with Python | Episode 11: Advanced Filtering and Efficient Extraction with BeautifulSoup

    In this lesson, you’ll learn about: advanced BeautifulSoup filtering techniques, custom extraction logic, real-world link scraping, and performance optimization using SoupStrainer1. Core Extraction Tools: find vs find_all🔹 The Basic Building Blocks🔹 What They DoMethodPurposefind()Returns first matchfind_all()Returns all matches🔹 Basic Examplefrom bs4 import BeautifulSoup import requests html = requests.get("https://example.com").text soup = BeautifulSoup(html, "lxml") soup.find("p") soup.find_all("a") 2. Filtering Beyond Tags🔹 Attribute-Based Selectionsoup.find_all("img", src=True) soup.find_all("a", id="main-link") 👉 Key Insight You’re no longer just finding tags—you’re filtering structured conditions3. Using Regular Expressions for Precision🔹 Regex in BeautifulSoupUse Regular Expressions for advanced filtering:import re soup.find_all("a", href=re.compile("wiki")) 🔹 What This Enables Match patterns in URLsFilter partial textDetect structured formats4. Custom Filtering Functions (Advanced Logic)🔹 When Built-ins Aren’t Enoughdef custom_filter(tag): return tag.has_attr("src") and not tag.has_attr("href") soup.find_all(custom_filter) 🔹 Real Use Cases Images without linksLinks pointing to specific domainsComplex multi-condition filtering👉 Key Insight You can encode any logic you want in Python5. Real-World Project: Scraping Links🔹 Target Site Workflow🔹 Step 1: Fetch Pageimport requests from bs4 import BeautifulSoup url = "https://mashable.com" html = requests.get(url).text soup = BeautifulSoup(html, "lxml") 🔹 Step 2: Extract Linkslinks = soup.find_all("a") 6. Absolute vs Relative URLs🔹 The ProblemTypeExampleAbsolutehttps://site.com/pageRelative/page🔹 Fixing Relative Linksfrom urllib.parse import urljoin full_url = urljoin(url, "/about") 👉 Key Insight Scrapers must normalize URLs for reliability7. Performance Optimization with SoupStrainer🔹 The IdeaInstead of parsing everything…👉 parse only what you need🔹 Implementationfrom bs4 import SoupStrainer, BeautifulSoup only_links = SoupStrainer("a") soup = BeautifulSoup(html, "lxml", parse_only=only_links) 🔹 Benefits Faster parsingLower memory usageCleaner output8. Mental Model🔹 Think Like This: HTML = giant datasetfind/find_all = SQL queriesregex = advanced filtering conditionsSoupStrainer = pre-filter at ingestionFinal TakeawayMastering Beautiful Soup is not just about extracting data—it’s about building a filtering system.Once you combine: Structural selectionRegex logicCustom filtersPerformance optimization👉 You can extract exactly what you want from any webpage efficiently and at scale. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 11: Advanced Filtering and Efficient Extraction with BeautifulSoup

  8. Jul 20

    Course 40 - Web Scraping with Python | Episode 10: Navigating and Extracting Web Data with Beautiful Soup

    In this lesson, you’ll learn about: how HTML is structured as a tree, how to turn raw pages into navigable data using Beautiful Soup, and how to extract specific elements efficiently1. Understanding the HTML Parse Tree🔹 The Structure of a Web PageEvery web page is a hierarchical tree made of nodes: Root → Children → and Siblings → elements at the same level🔹 Key Sections → metadata (title, scripts, styles) → visible content👉 Key Insight Scraping is really about navigating this tree intelligently2. Turning HTML into Data (Beautiful Soup)🔹 The Core ToolUse Beautiful Soup Converts raw HTML → structured Python objectMakes navigation simple and readable🔹 Why It’s Powerful Handles messy HTMLSupports multiple parsersEasy to search and extract3. Choosing the Right Parser🔹 Available ParsersParserStrengthlxmlFast and efficienthtml5libHandles broken HTML🔹 When to Use Each Use lxml → performanceUse html5lib → unreliable or malformed pages👉 Pro Insight Real-world pages are often messy → parser choice matters4. From Request to Parsed Tree🔹 Workflow Overview Send HTTP requestReceive HTMLParse with Beautiful SoupNavigate and extract🔹 Example Setupimport requests from bs4 import BeautifulSoup r = requests.get("https://example.com") soup = BeautifulSoup(r.text, "lxml") 5. Extracting Text Content🔹 Headers & Paragraphstitle = soup.h1.string paragraph = soup.p.string 👉 Use Case Blog titlesArticle contentProduct descriptions6. Extracting Attributes (Links & Images)🔹 Accessing Attributeslink = soup.a["href"] image = soup.img["src"] 👉 What You Can Extract URLsImage sourcesMetadata7. Working with CSS Classes🔹 Finding Elements by Classitems = soup.find_all("div", class_="product") 🔹 Important Note Classes can be multi-valued 👉 Beautiful Soup handles this intelligently8. Navigating the Tree🔹 Moving Through Nodes .parent.children.next_sibling🔹 Examplefor child in soup.body.children: print(child) 👉 Key Skill Understanding relationships = better extraction9. Real Extraction Strategy🔹 Step-by-Step Thinking Inspect HTMLIdentify target elementChoose selectorExtract dataClean output10. Common Pitfalls🔹 Things to Watch Out For Missing tagsNested complexityDynamic content (JavaScript)👉 Solution Always verify structure firstUse browser DevTools11. Mental ModelHTML Page = Tree Beautiful Soup = Navigator👉 You are not scraping randomly You are walking a structured mapFinal TakeawayMastering Beautiful Soup means mastering how the web is structured.Once you understand the tree, extraction becomes predictable, scalable, and precise—turning messy HTML into clean, usable data. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

    Course 40 - Web Scraping with Python | Episode 10: Navigating and Extracting Web Data with Beautiful Soup
See All (311)

About

Welcome to CyberCode Academy — your audio classroom for Programming and Cybersecurity. 🎧 Each course is divided into a series of short, focused episodes that take you from beginner to advanced level — one lesson at a time. From Python and web development to ethical hacking and digital defense, our content transforms complex concepts into simple, engaging audio learning. Study anywhere, anytime — and level up your skills with CyberCode Academy. 🚀 Learn. Code. Secure. You can listen and download our episodes for free on more than 10 different platforms: https://linktr.ee/cybercode_academy

Information

  • Creator
    CyberCode Academy
  • Years Active
    2025 - 2026
  • Episodes
    311
  • Rating
    Clean
  • Copyright
    © Copyright CyberCode Academy
  • Show Website
    CyberCode Academy

You Might Also Like