Python Regular expresions full guide

Regular expressions (regex) are a powerful tool for manipulating and analyzing text. In Python, we use the re module to work with regex.

import re # No need to pip install its in the standard library

Patterns

Basic Characters

• a: Exact match. re.search('a', 'apple')
• .: Matches any character (except newline). re.search('.', 'apple')
• \d: Matches any digit (0-9). re.search('\d', 'apple2')
• \D: Matches any non-digit. re.search('\D', '1234a')
• \s: Matches any whitespace. re.search('\s', 'apple pie')
• \S: Matches any non-whitespace. re.search('\S', ' apple')
• \w: Matches any alphanumeric character and underscore (a-z, A-Z, 0-9, _). re.search('\w', '@apple!')
• \W: Matches any non-alphanumeric character. re.search('\W', 'apple@')

Special Characters

• \t: Tab. re.search('\t', 'apple\t')
• \n: Newline. re.search('\n', 'apple\npie')
• \r: Carriage Return. re.search('\r', 'apple\r\n')
• \\: Backslash. re.search('\\\\', 'apple\\')

Quantifiers

• *: Zero or more of the previous item. re.search('a*', 'aaapple')
• +: One or more of the previous item. re.search('a+', 'aaapple')
• ?: Zero or one of the previous item. re.search('a?', 'aaapple')
• {n}: Exactly n of the previous item. re.search('a{2}', 'aaapple')
• {n,}: n or more of the previous item. re.search('a{2,}', 'aaapple')
• {,m}: Up to m of the previous item. re.search('a{,2}', 'aaapple')
• {n,m}: Between n and m of the previous item. re.search('a{2,3}', 'aaaapple')

Groups and Ranges

• [abc]: Matches any of the enclosed characters. re.search('[abc]', 'apple')
• [^abc]: Matches any character not enclosed. re.search('[^abc]', 'apple')
• (abc): Defines a group. re.search('(abc)', 'abcapple')
• (a|b): Matches either a or b. re.search('(a|p)', 'apple')

Anchors

• ^abc: Matches pattern abc at the start of a string. re.search('^abc', 'abcapple')
• abc$: Matches pattern abc at the end of a string. re.search('abc$', 'appleabc')
• \babc: Word boundary (matches abc at the start of a word). re.search('\\babc', 'abc apple')
• abc\b: Word boundary (matches abc at the end of a word). re.search('abc\\b', 'appleabc pie')

Flags

• re.I or re.IGNORECASE: Makes matching case insensitive. re.search('a', 'APPLE', re.I)
• re.M or re.MULTILINE: Makes ^ and $ match start and end of each line. re.search('^a', 'apple\nbanana', re.M)
• re.S or re.DOTALL: Makes . match any character, including newlines. re.search('a.p', 'a\np', re.S)
• re.X or re.VERBOSE: Allows multiline regular expressions and ignores whitespace and comments in the pattern. re.search("""a # this is a comment\nb""", 'ab', re.X)

Back References

Backreferences in a pattern allow you to specify that the contents of an earlier capturing group must also be found at the current location in the string.

• \1: Matches the contents of group 1. re.search('(a)b\\1', 'aba')
• \2: Matches the contents of group 2. re.search('(a)(b)\\2', 'abb')

Lookahead and Lookbehind

Lookahead and lookbehind assertions determine the success or failure of a regex match in Python based on what is just behind (to the left) or ahead (to the right) of the current string position.

• a(?=b): Positive lookahead: Matches 'a' only if 'a' is followed by 'b'. re.search('a(?=b)', 'ab')
• a(?!b): Negative lookahead: Matches 'a' only if 'a' is not followed by 'b'. re.search('a(?!b)', 'ac')
• (?<=b)a: Positive lookbehind: Matches 'a' only if 'a' is preceded by 'b'. re.search('(?<=b)a', 'ba')
• (?<!b)a: Negative lookbehind: Matches 'a' only if 'a' is not preceded by 'b'. re.search('(?<!b)a', 'ca')

Python’s re module

Python’s re module provides several functions to work with regex. Here are the most used beyondre.search():

re.match()

This function checks for a match only at the beginning of the string.

print(re.match('abc', 'abcdef'))  # <re.Match object; span=(0, 3), match='abc'>
print(re.match('abc', 'abcdefabc'))  # <re.Match object; span=(0, 3), match='abc'>
print(re.match('abc', 'abcdefabc').group())  # abc

re.findall()

This function returns all non-overlapping matches of pattern in string, as a list of strings.

print(re.findall('abc', 'abcdefabc'))  # ['abc', 'abc']

re.sub()

This function replaces all occurrences of the RE pattern in string with repl, substituting all occurrences unless max provided.

print(re.sub('abc', '123', 'abcdefabc'))  # 123def123

re.split()

This function splits the source string by the occurrences of the pattern.

print(re.split('\d+', 'apple123banana45cherry6'))  # ['apple', 'banana', 'cherry', '']

Popular Examples

import re

# Find all substrings that match a pattern
text = "Hello, my name is John Doe. I live in New York."
matches = re.findall(r'\b\w{4}\b', text)  
# matches: ['Hello', 'name', 'John', 'live', 'York']
# This code finds all 4-letter words in the text.

# Split a string by multiple delimiters
text = "apple;banana-orange:peach"
result = re.split(r'[;:-]', text)  
# result: ['apple', 'banana', 'orange', 'peach']
# This code splits the text by either a semicolon, a dash, or a colon.

# Replace substrings that match a pattern
text = "I have 3 cats, 4 dogs, and 5 fishes."
result = re.sub(r'\d', 'many', text)  
# result: 'I have many cats, many dogs, and many fishes.'
# This code replaces all digits in the text with the word 'many'.

# Check if a string starts with a pattern
text = "Hello, world!"
result = bool(re.match(r'^Hello', text))  
# result: True
# This code checks if the text starts with 'Hello'.

# Extract email addresses from a string
text = "Contact us at info@example.com or support@example.net."
emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', text)  
# emails: ['info@example.com', 'support@example.net']
# This code extracts all email addresses from the text.

# Find all dates in YYYY-MM-DD format
text = "I was born on 2000-01-01. I graduated on 2020-05-15."
dates = re.findall(r'\b\d{4}-\d{2}-\d{2}\b', text)  
# dates: ['2000-01-01', '2020-05-15']
# This code extracts all dates in YYYY-MM-DD format from the text.

# Capture groups in a match
text = "The event will be held on 2023-07-10 at 18:00."
match = re.search(r'(\d{4}-\d{2}-\d{2}) at (\d{2}:\d{2})', text)
date, time = match.groups()
# date: '2023-07-10', time: '18:00'
# This code extracts the date and time from the text.

# Match a pattern multiple times
text = "I love apples, apples are my favorite fruit."
matches = re.findall(r'(apples)', text)  
# matches: ['apples', 'apples']
# This code finds all occurrences of 'apples' in the text.

# Match a pattern and replace it with a function's result
def replace_with_length(match):
    return str(len(match.group()))

text = "I have a cat, a dog, and a horse."
result = re.sub(r'\ba \w+?\b', replace_with_length, text)  
# result: 'I have 1 cat, 1 dog, and 1 horse.'
# This code replaces all 'a [word]' with the length of '[word]'.

# Match nested brackets correctly
text = "foo(bar(baz))blim"
matches = re.findall(r'\(([^()]*)\)', text)  
# matches: ['baz']
# This code finds all text within the innermost brackets.

# Find duplicate words
text = "This is is a test test sentence."
dupes = re.findall(r'\b(\w+)\s+\1\b', text)  
# dupes: ['is', 'test']
# This code finds all duplicate words in the text.

# Match a pattern except in specific contexts
text = "100 dollars, but not 100 cents"
matches = re.findall(r'100(?!\s+cents)', text)  
# matches: ['100']
# This code finds '100' except when it is followed by ' cents'.

# Match balanced parentheses
text = "((()))()()(((())))"
matches = re.findall(r'\(([^()]|(?R))*\)', text)  
# matches: ['((()))', '()', '(((())))']
# This code matches balanced parentheses in the text.

# Validate a password with certain rules
password = "StrongPass1!"
is_valid = bool(re.match(r'^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*]).{8,}$', password))  
# is_valid: True
# This code validates the password, which must contain at least one digit, one lowercase letter, one uppercase letter, one special character, and be at least 8 characters long.

# Extract the domain name from a URL
url = "https://www.example.com/path?query#fragment"
domain = re.search(r'https?://([A-Za-z_0-9.-]+).*', url).group(1)  
# domain: 'www.example.com'
# This code extracts the domain name from a URL.

# Match a Unicode character
text = "Résumé"
matches = re.findall(r'\w+', text)  
# matches: ['Résumé']
# This code finds all words in the text, even if they contain Unicode characters.

# Match repeating words
text = "This is a a test."
repeated_words = re.findall(r'\b(\w+)\s+\1\b', text)  
# repeated_words: ['a']
# This code finds all words that are immediately repeated.

# Match words that are palindromes
text = "A man, a plan, a canal, Panama"
palindromes = [word for word in re.findall(r'\b\w+\b', text) if word == word[::-1]]
# palindromes: ['A', 'man', 'a', 'a', 'Panama']
# This code finds all palindromes in the text.

# Match words containing 'q' not followed by 'u'
text = "Iraq is a country in the Middle East."
q_not_u_words = re.findall(r'\b\w*q[^u]\w*\b', text)  
# q_not_u_words: ['Iraq']
# This code finds all words in the text that contain 'q' not followed by 'u'.

# Extract all words within double quotes
text = 'He said, "Hello, world!"'
quoted = re.findall(r'"(.*?)"', text)  
# quoted: ['Hello, world!']

# This code extracts all words within double quotes from the text. 

https://medium.com/@theom/the-ultimate-python-regex-cheat-sheet-f202e99ac21d

summary of the information about different data structures in

Here's an organized summary of the information about different data structures in Python, along with methods to iterate over them as key-value pairs when applicable:

Feature by Data Structure

Feature	Dictionary	Set	List	String	Tuple
Definition	Stores key pairs	An unordered collection of unique elements	A sequential, mutable collection of any data type	A sequential, immutable collection of textual data	A sequential, immutable collection of any data type
Representation	{ 'a':[42], 'b':[23,6,1] }	{'^2', 'mc', 'equal', 'E'}	[ 'a','b', 3, 4 ]	"call me ishmael"	( 'commander','lambda')
How to create?	x = {}, x = dict()	x = set()	x = [], x = list()	x = "", x = str()	x = ('a','b',), x = tuple()
Is structure mutable and allow duplicate elements?	Immutable keys but mutable and duplicate values	Mutable but unique elements only	Mutable and allows duplicate elements	Immutable but allows duplicate elements	Immutable but allows duplicate elements
Is the structure iterable?	Yes (iterable over keys)	Yes	Yes	Yes	Yes

Methods and Iteration

Dictionary

• Methods: items(), keys(), values()
• Iteration:

  python
  d = { 'a': [42], 'b': [23, 6, 1] }
  for key, value in d.items():
      print(f"{key}: {value}")
  

Set

• Methods: No key-value methods, primarily add(), remove(), union(), etc.
• Iteration:

  python
  s = {'^2', 'mc', 'equal', 'E'}
  for element in s:
      print(element)
  

List

• Methods: Index-based methods, append(), extend(), insert(), remove(), etc.
• Iteration:

  python
  l = ['a', 'b', 3, 4]
  for index, value in enumerate(l):
      print(f"{index}: {value}")
  

String

• Methods: String-specific methods, split(), join(), find(), replace(), etc.
• Iteration:

  python
  s = "call me ishmael"
  for index, char in enumerate(s):
      print(f"{index}: {char}")
  

Tuple

• Methods: Tuple-specific methods, mainly count() and index()
• Iteration:

  python
  t = ('commander', 'lambda')
  for index, value in enumerate(t):
      print(f"{index}: {value}")
  

Summary

• Dictionary: Best for key-value paired data, mutable values, and iterable over keys.
• Set: Ideal for unique, unordered data, mutable, but no key-value pairs.
• List: Great for ordered, mutable collections with duplicates allowed.
• String: Immutable text data, iterated as characters.
• Tuple: Immutable ordered collections, can contain duplicates.

google speech recog from uri for audio longer than 1 minute

 from google.cloud import speech

from google.oauth2 import service_account  # Import for service account credentials

def run_quickstart(audio_uri: str) -> speech.RecognizeResponse:

    """Transcribes audio from a Google Cloud Storage URI using the Speech-to-Text API.

    Args:

        audio_uri (str): The URI of the audio file in Google Cloud Storage.

    Returns:

        speech.RecognizeResponse: The response object containing the transcription results.

    Raises:

        RuntimeError: If an error occurs during transcription or credential setup.

    """

    # Explicit credential handling

    credential_path = "/home/ambiorixg12/mycodes/google_speech/voice.json"  # Replace with your actual path

    try:

        credentials = service_account.Credentials.from_service_account_file(credential_path)

        print(f"Using Explicit Credentials from {credential_path}")

    except Exception as e:

        print(f"Error loading explicit credentials: {e}")

        raise RuntimeError("Failed to load explicit credentials")

    # Instantiates a client (use credentials)

    client = speech.SpeechClient(credentials=credentials)

    try:

        # The name of the audio file to transcribe

        audio = speech.RecognitionAudio(uri=audio_uri)

        config = speech.RecognitionConfig(

            encoding=speech.RecognitionConfig.AudioEncoding.LINEAR16,

            sample_rate_hertz=16000,

            language_code="en-US",

        )

        # Detects speech in the audio file

        response = client.recognize(config=config, audio=audio)

        for result in response.results:

            print(f"Transcript: {result.alternatives[0].transcript}")

        return response

    except Exception as e:

        print(f"Transcription error: {e}")

        raise RuntimeError("Error occurred during transcription")

if __name__ == "__main__":

    audio_file_path = "gs://cloud-samples-data/speech/brooklyn_bridge.raw"

    run_quickstart(audio_file_path)

https://cloud.google.com/speech-to-text/docs/async-recognize#:~:text=Attempting%20to%20transcribe%20local%20audio,the%20operation%20using%20the%20google.

https://codelabs.developers.google.com/codelabs/cloud-speech-text-python3#3

google python speech recog

 import argparse

import os

from google.cloud import speech_v1p1beta1

from google.oauth2 import service_account

def transcribe_file(speech_file: str, credentials_file: str) -> speech_v1p1beta1.types.RecognizeResponse:

    """Transcribe the given audio file."""

    # Authentication

    credentials = service_account.Credentials.from_service_account_file(credentials_file)

    client = speech_v1p1beta1.SpeechClient(credentials=credentials)

    with open(speech_file, "rb") as audio_file:

        content = audio_file.read()

    audio = speech_v1p1beta1.RecognitionAudio(content=content)

    config = speech_v1p1beta1.RecognitionConfig(

        encoding=speech_v1p1beta1.RecognitionConfig.AudioEncoding.LINEAR16,

        sample_rate_hertz=8000,

        language_code="en-US",

    )

    response = client.recognize(config=config, audio=audio)

    # Print the transcription

    for result in response.results:

        print(f"Transcript: {result.alternatives[0].transcript}")

    return response

# Execute the transcription function

if __name__ == "__main__":

    parser = argparse.ArgumentParser(description="Transcribe audio file using Google Cloud Speech-to-Text API")

    parser.add_argument("speech_file", help="Path to the audio file to transcribe")

    parser.add_argument("--credentials_file", default="/home/ambiorixg12/mycodes/google_speech/voice.json", help="Path to the JSON file containing Google Cloud credentials")

    args = parser.parse_args()

    transcribe_file(args.speech_file, args.credentials_file)

import assemblyai transcription

 import assemblyai as aai

import sys

# set the API key

ASSEMBLYAI_API_KEY="231d38"

aai.settings.api_key = f"{ASSEMBLYAI_API_KEY}"

transcriber = aai.Transcriber()

transcript = transcriber.transcribe(sys.argv[1])

print(transcript.text)