comparison test_jsscan.py @ 4:30c1f55eff96

fixed greedy regexp bug
author Atul Varma <avarma@mozilla.com>
date Thu, 22 Apr 2010 17:31:32 -0700
parents f82ff2c61c06
children
comparison
equal deleted inserted replaced
3:ce894f57b30c 4:30c1f55eff96
23 Filtering: 23 Filtering:
24 24
25 >>> tokenize(' k', ignore='whitespace') 25 >>> tokenize(' k', ignore='whitespace')
26 ('name', 'k', (1, 2)) 26 ('name', 'k', (1, 2))
27 27
28 Many double-quoted strings on the same line:
29
30 >>> tokenize(r'"hello there "+" dude"')
31 ('string', '"hello there "', (1, 0))
32 ('literal', '+', (1, 14))
33 ('string', '" dude"', (1, 15))
34
35 Many single-quoted strings on the same line:
36
37 >>> tokenize(r"'hello there '+' dude'")
38 ('string', "'hello there '", (1, 0))
39 ('literal', '+', (1, 14))
40 ('string', "' dude'", (1, 15))
41
28 Escaped double-quoted strings: 42 Escaped double-quoted strings:
29 43
30 >>> tokenize(r'"i say \\"tomato\\""') 44 >>> tokenize(r'"i say \\"tomato\\""')
31 ('string', '"i say \\\\"tomato\\\\""', (1, 0)) 45 ('string', '"i say \\\\"tomato\\\\""', (1, 0))
32 46