HTML Bitcoin address parsing using Beautiful soup

Question

I'm trying to use the beautiful soup lib to parse a webpage and find bitcoin addresses.

I've managed to pull the class containing a generated address out of the whole html document:

<div class="roundpic qrcode" data-height="80" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="80" style="margin: auto"></div>, <div class="roundpic qrcode" data-height="160" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="160" style="padding: 10px"></div>

What would be the best way to isolate the address? I know the length can be between 27-34 characters but it will always appear in between 'bitcoin:' and '?'. Is there a regex I could use?

slayer · Answer 1 · Sep 5, 2018

You don't really need a regex. Basic string operations work just fine:

import re
from bs4 import BeautifulSoup

html = '''
<div class="roundpic qrcode" data-height="80" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="80" style="margin: auto"></div>
<div class="roundpic qrcode" data-height="160" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="160" style="padding: 10px"></div>
'''

soup = BeautifulSoup(html)

for div in soup.find_all('div', {'data-text': re.compile(r'^bitcoin:')}):
    address, amount = div.get('data-text').replace('bitcoin:', '').split('?amount=')

answered Sep 5, 2018 by slayer
• 29,370 points

score 0 · Answer 2 · Sep 5, 2018

This is very easy using python regular expressions:

import re
in_str=<div class="roundpic qrcode" data-height="80" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="80" style="margin: auto"></div>, <div class="roundpic qrcode" data-height="160" data-text="bitcoin:1JL7kugm1vDLqyzrVPAPdcbjH3PTxcPcud?amount=0.0573" data-width="160" style="padding: 10px"></div>
bitcoin_addresses=re.findall(r'bitcoin:(.*?)\?amount',in_str)

answered Sep 5, 2018 by Him

score 0 · Answer 3 · Sep 5, 2018

Regex is not meant for parsing. As you have already parsed the DOM anyway, why don't you access it directly? :)

# imports
from bs4 import BeautifulSoup as Soup

# parse the HTML
s = Soup("<html><body><div data-text='bitcoin:a'></div><div data-text='bitcoin:b'></div><div data-text='bitcoin:c'></div></body></html>")

# find the divs with "data-text"-attribute
divs = ( d for d in s.findAll(name="div") if d.has_key('data-text') )

# extract the value of "data-text"
data_texts = map(lambda x: x["data-text"], divs)

# find only bitcoins
bitcoins = filter(lambda s: s.startswith("bitcoin:"), data_texts)

# strip the prefix
extract = map(lambda s: s[8:], bitcoins)

# result
print extract