PostDataToURL

[xjonnie]

Need help with this function.

The goal is to bypass the web form and load the resulting table direct to a field.

Modified the PostDataToURL function to include the additional parameters, just hardcoded.

The script returns ERROR.

FM11 Adv, MAC

Thanks in advance!

Calling script.

PostDataToURLxx( "FFX" ; "2" ; http://aa.usno.navy.mil/cgi-bin/aa_rstablew.pl )

Registration script.

RegisterGroovy( "PostDataToURLxx( key ; value ; url )" ; "// Construct data¶

String data = URLEncoder.encode(key, \"UTF-8\") + \"=\" + URLEncoder.encode(value, \"UTF-8\");¶

// use something like the following to send multiple key/value pairs¶

data += \"&\" + URLEncoder.encode(xxy, \"UTF-8\") + \"=\" + URLEncoder.encode(2012, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(type, \"UTF-8\") + \"=\" + URLEncoder.encode(0, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(place, \"UTF-8\") + \"=\" + URLEncoder.encode(Sparta, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(xx0, \"UTF-8\") + \"=\" + URLEncoder.encode(-1, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(xx1, \"UTF-8\") + \"=\" + URLEncoder.encode(93, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(xx2, \"UTF-8\") + \"=\" + URLEncoder.encode(6, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(yy0, \"UTF-8\") + \"=\" + URLEncoder.encode(1, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(yy1, \"UTF-8\") + \"=\" + URLEncoder.encode(37, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(yy2, \"UTF-8\") + \"=\" + URLEncoder.encode(2, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(zz1, \"UTF-8\") + \"=\" + URLEncoder.encode(6, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(zz0, \"UTF-8\") + \"=\" + URLEncoder.encode(-1, \"UTF-8\");¶

data += \"&\" + URLEncoder.encode(ZZZ, \"UTF-8\") + \"=\" + URLEncoder.encode(END, \"UTF-8\")¶

¶

// Send data¶

URL url = new URL(url);¶

URLConnection conn = url.openConnection();¶

conn.setDoOutput(true);¶

OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());¶

wr.write(data);¶

wr.flush();¶

¶

// Get the response¶

String response = conn.getInputStream().getText(\"utf-8\");¶

wr.close();¶

return response;"; "isGui=false" )

= "ERROR"

The website with the form.

http://aa.usno.navy..../RS_OneYear.php

Here is just the form from the source.

<a name="formb"></a>

<h3 style="color: red; border-top: 1px solid black; margin-top: 1em; padding-top: 1em;">Form B - Locations Worldwide</h3>

<h2>Specify year, type of table, and place:</h2>

<form method="post" target="_parent" action="http://aa.usno.navy....a_rstablew.pl">

<input type="hidden" name="FFX" value="2">

<p>

<span class="bold"><label for="xxy_int">Year:</label></span>&nbsp;&nbsp;<input type="text" name="xxy" id="xxy_int" size="4" maxlength="4" value="2012">

<label for="tabtyb"><b>Type of table:</b></label> <select name="type" id="tabtyb">

<option selected value="0">sunrise/sunset</option>

<option value="1">moonrise/moonset</option>

<option value="2">civil twilight</option>

<option value="3">nautical twilight</option>

<option value="4">astronomical twilight</option>

</select></p>

<span class="bold"><label for="placeint">Place Name Label:</label></span> &nbsp;

<input type="text" name="place" id="placeint" size="32" maxlength="32" value="(no name given)">

<p>The place name you enter above is merely a label for the table header;

you can enter any identifier, or none (avoid using punctuation characters).

The data will be calculated for the longitude and latitude you enter below. </p>

<fieldset><legend><b>Longitude:</b></legend>

<input type="radio" name="xx0" id="east" value="1"> <label for="east">east</label>&nbsp;

<input type="radio" name="xx0" id="west" checked value="-1"> <label for="west">west</label>&nbsp;

<input type="text" name="xx1" id="dego" size="3" maxlength="3" value=""> <label for="dego">degrees</label>&nbsp;

<input type="text" name="xx2" id="mino" size="2" maxlength="2" value=""> <label for="mino">minutes</label>&nbsp;

</fieldset>

<fieldset><legend><b>Latitude:</b></legend>

<input type="radio" name="yy0" id="north" checked value="1"> <label for="north">north</label>&nbsp;

<input type="radio" name="yy0" id="south" value="-1"> <label for="south">south</label>&nbsp;

<input type="text" name="yy1" id="dega" size="2" maxlength="2" value=""> <label for="dega">degrees</label>&nbsp;

<input type="text" name="yy2" id="mina" size="2" maxlength="2" value=""> <label for="mina">minutes</label>&nbsp;

</fieldset>

<fieldset><legend><b>Time Zone:</b></legend>

<input type="text" name="zz1" id="zhour" size="5" maxlength="5"> <label for="zhour">hours</label>&nbsp;

<input type="radio" name="zz0" id="zeast" value="1"> <label for="zeast">east of Greenwich</label>&nbsp;

<input type="radio" name="zz0" id="zwest" checked value="-1"> <label for="zwest">west of Greenwich</label>

</fieldset>

<p>For locations that require it, the time zone can be entered in hours and a

fraction. For example, for locations in India, the time zone may be entered

as 5.5 hours east of Greenwich. The time zone field can accommodate up to

five characters.</p>

Need coordinates?&nbsp; Try NGA's

<a href="http://earth-info.ng...tml/index.html" target="blank">GEOnet

Names Server (GNS)</a>.

<br>Need U.S. coordinates?&nbsp; Try the USGS

<a href="http://geonames.usgs...ls/gnispublic/" target="blank">Geographic Names Information System (GNIS)</a>.

<br>

Need a time zone?&nbsp; Try the <a href="faq/docs/world_tzones.php">time zone map</a>.

<input type="hidden" name="ZZZ" value="END">

<p><input type="submit" value="Compute Table">

<input type="reset" value="Clear all fields"></p>

</form>

[john renfrew]

Do you have an example of a url string which does work?

Why are you calling URLencode a million times?

And if you move to URLEncoder.encode(-1, 'UTF-8') you will also find the end code MUCH simpler to troubleshoot.

PostDataToURLxx( "FFX" ; "2" ; "http://aa.usno.navy.mil/cgi-bin/aa_rstablew.pl" )??

And the data string is unlikely to work without some quotes

String data = URLEncoder.encode(key, 'UTF-8') + '=' + URLEncoder.encode(value, 'UTF-8');

// use something like the following to send multiple key/value pairs

data += '&' + URLEncoder.encode('xxy', 'UTF-8') + '=' + URLEncoder.encode('2012', 'UTF-8');

data += '&' + URLEncoder.encode('type', 'UTF-8') + '=' + URLEncoder.encode('0', 'UTF-8');

data += '&' + URLEncoder.encode('place', 'UTF-8') + '=' + URLEncoder.encode('Sparta', 'UTF-8');

data += '&' + URLEncoder.encode('xx0', 'UTF-8') + '=' + URLEncoder.encode('-1', 'UTF-8');

data += '&' + URLEncoder.encode('xx1', 'UTF-8') + '=' + URLEncoder.encode('93', 'UTF-8');

data += '&' + URLEncoder.encode('xx2', 'UTF-8') + '=' + URLEncoder.encode('6', 'UTF-8');

data += '&' + URLEncoder.encode('yy0', 'UTF-8') + '=' + URLEncoder.encode('1', 'UTF-8');

data += '&' + URLEncoder.encode('yy1', 'UTF-8') + '=' + URLEncoder.encode('37', 'UTF-8');

data += '&' + URLEncoder.encode('yy2', 'UTF-8') + '=' + URLEncoder.encode('2', 'UTF-8');

data += '&' + URLEncoder.encode('zz1', 'UTF-8') + '=' + URLEncoder.encode('6', 'UTF-8');

data += '&' + URLEncoder.encode('zz0', 'UTF-8') + '=' + URLEncoder.encode('-1', 'UTF-8');

data += '&' + URLEncoder.encode('ZZZ', 'UTF-8') + '=' + URLEncoder.encode('END', 'UTF-8'

)

but his will give the same string as you appear to not have any odd characters in your values

String data = key + '=' + value;

// use something like the following to send multiple key/value pairs

data += '&' + 'xxy' + '=' + '2012';

data += '&' + 'type' + '=' + '0';

data += '&' + 'place' + '=' + 'Sparta';

data += '&' + 'xx0' + '=' + '-1';

data += '&' + 'xx1' + '=' + '93';

data += '&' + 'xx2' + '=' + '6';

data += '&' + 'yy0' + '=' + '1';

data += '&' + 'yy1' + '=' + '37';

data += '&' + 'yy2' + '=' + '2';

data += '&' + 'zz1' + '=' + '6';

data += '&' + 'zz0' + '=' + '-1';

data += '&' + 'ZZZ' + '=' + 'END'

if you pass the keys and values from FM as a List("key"; 'key2") and values as List(val ; val2)

then this will iterate through them and make it easier to adapt to a different number of keys

String data = 'key' + '=' + 'value'

dk = dataKeys.tokenize('n')

dv = dataValues.tokenize('n')

i = 0

dk.each(){

data += '&' + it +'=' + dv[i]

i++

}

[xjonnie]

Thanks for your response!

Do you have an example of a url string which does work?

Why are you calling URLencode a million times?

And if you move to URLEncoder.encode(-1, 'UTF-8') you will also find the end code MUCH simpler to troubleshoot.

PostDataToURLxx( "FFX" ; "2" ; "http://aa.usno.navy.mil/cgi-bin/aa_rstablew.pl" )??

And the data string is unlikely to work without some quotes

Dont have a sample.

All I know is looking at the source http://aa.usno.navy..../RS_OneYear.php

No api or info is provided.

URLEncode a million times

The code is from the sample in scriptmaster which suggests this is the way to add parameters.

PostDataToURLxx( "FFX" ; "2" ; "http://aa.usno.navy.mil/cgi-bin/aa_rstablew.pl" )??

from the source

form method="post" target="_parent" action="http://aa.usno.navy.mil/cgi-bin/aa_rstablew.pl"

Note: if I call with just

RegisterGroovy( "PostDataToURLxx( key ; value ; url )" ; "// Construct data¶

String data = URLEncoder.encode(key, "UTF-8") + "=" + URLEncoder.encode(value, "UTF-8");

...

I do get a response

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<title>Sun or Moon Rise/Set Table for One Year</title>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

</head>

<body>

<pre>

Input error. One of your input values is in the wrong format, blank, or out of bounds.

</pre>

<p><a href="javascript:history.go(-1)">Back to form</a></p>

</body>

</html>

[john renfrew]

Hmmm

It does actually work though

PostDataToURL ( key ; value ; url ; dataKeys ; dataValues )

With parameters:

key: FFX

value: 2

url: http://aa.usno.navy..../aa_rstablew.pl

dataKeys: although you Don't need the ZZZ and END Or the TimeZone values Or the placeName

xxy

type

place

xx0

xx1

xx2

yy0

yy1

yy2

zz1

zz0

ZZZ

dataValues:

2012

0

Sparta

-1

93

6

1

37

2

6

-1

END

RESULT

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<title>Sun or Moon Rise/Set Table for One Year</title>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

</head>

<body>

<pre>

			 o , o ,									 SPARTA								 Astronomical Applications Dept.

Location: W093 06, N37 02						 Rise and Set for the Sun for 2012				 U. S. Naval Observatory	

																									 Washington, DC 20392-5420	

													 Zone: 6h West of Greenwich													

																																	

																																	

	 Jan.	 Feb.	 Mar.	 Apr.	 May	 June	 July	 Aug.	 Sept.	 Oct.	 Nov.	 Dec.

Day Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set Rise Set

	 h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m h m

01 0726 1706 0715 1737 0643 1807 0557 1835 0517 1902 0454 1927 0457 1936 0518 1919 0544 1840 0608 1755 0637 1714 0708 1656

02 0726 1707 0714 1738 0641 1808 0556 1836 0516 1903 0454 1928 0457 1936 0519 1918 0544 1839 0609 1753 0638 1713 0708 1655

03 0726 1708 0714 1739 0640 1809 0554 1837 0515 1904 0453 1928 0458 1936 0520 1917 0545 1837 0610 1752 0639 1712 0709 1655

04 0726 1708 0713 1740 0638 1810 0553 1838 0514 1905 0453 1929 0458 1936 0520 1916 0546 1836 0611 1750 0640 1711 0710 1655

05 0726 1709 0712 1741 0637 1811 0552 1839 0513 1905 0453 1929 0459 1935 0521 1915 0547 1834 0612 1749 0641 1710 0711 1655

06 0726 1710 0711 1742 0636 1812 0550 1840 0512 1906 0453 1930 0459 1935 0522 1914 0548 1833 0612 1748 0642 1709 0712 1655

07 0726 1711 0710 1743 0634 1813 0549 1841 0511 1907 0453 1930 0500 1935 0523 1913 0548 1831 0613 1746 0643 1709 0713 1655

08 0726 1712 0709 1745 0633 1814 0547 1842 0510 1908 0452 1931 0500 1935 0524 1912 0549 1830 0614 1745 0644 1708 0714 1655

09 0726 1713 0708 1746 0631 1815 0546 1842 0509 1909 0452 1931 0501 1934 0525 1910 0550 1828 0615 1743 0645 1707 0714 1656

10 0726 1714 0707 1747 0630 1816 0544 1843 0508 1910 0452 1932 0502 1934 0525 1909 0551 1827 0616 1742 0646 1706 0715 1656

11 0726 1715 0706 1748 0628 1817 0543 1844 0507 1911 0452 1932 0502 1933 0526 1908 0552 1825 0617 1740 0647 1705 0716 1656

12 0726 1716 0705 1749 0627 1817 0542 1845 0506 1912 0452 1933 0503 1933 0527 1907 0553 1824 0618 1739 0648 1704 0717 1656

13 0725 1717 0704 1750 0626 1818 0540 1846 0506 1912 0452 1933 0504 1933 0528 1906 0553 1822 0619 1738 0650 1704 0717 1656

14 0725 1718 0703 1751 0624 1819 0539 1847 0505 1913 0452 1934 0504 1932 0529 1905 0554 1821 0620 1736 0651 1703 0718 1657

15 0725 1719 0702 1752 0623 1820 0537 1848 0504 1914 0452 1934 0505 1932 0530 1903 0555 1819 0621 1735 0652 1702 0719 1657

16 0725 1720 0700 1753 0621 1821 0536 1849 0503 1915 0452 1934 0506 1931 0530 1902 0556 1818 0622 1734 0653 1702 0719 1657

17 0724 1721 0659 1754 0620 1822 0535 1849 0502 1916 0452 1935 0506 1931 0531 1901 0557 1816 0622 1732 0654 1701 0720 1658

18 0724 1722 0658 1755 0618 1823 0533 1850 0502 1917 0453 1935 0507 1930 0532 1859 0557 1815 0623 1731 0655 1700 0721 1658

19 0723 1723 0657 1756 0617 1824 0532 1851 0501 1917 0453 1935 0508 1929 0533 1858 0558 1813 0624 1730 0656 1700 0721 1658

20 0723 1724 0656 1757 0615 1825 0531 1852 0500 1918 0453 1935 0509 1929 0534 1857 0559 1812 0625 1728 0657 1659 0722 1659

21 0723 1725 0654 1758 0614 1826 0530 1853 0500 1919 0453 1936 0509 1928 0535 1856 0600 1810 0626 1727 0658 1659 0722 1659

22 0722 1726 0653 1759 0612 1826 0528 1854 0459 1920 0453 1936 0510 1927 0535 1854 0601 1808 0627 1726 0659 1658 0723 1700

23 0721 1727 0652 1800 0611 1827 0527 1855 0458 1921 0454 1936 0511 1927 0536 1853 0601 1807 0628 1725 0700 1658 0723 1700

24 0721 1728 0651 1801 0609 1828 0526 1856 0458 1921 0454 1936 0512 1926 0537 1851 0602 1805 0629 1723 0701 1658 0724 1701

25 0720 1729 0649 1802 0608 1829 0525 1857 0457 1922 0454 1936 0512 1925 0538 1850 0603 1804 0630 1722 0702 1657 0724 1702

26 0720 1730 0648 1803 0606 1830 0523 1857 0457 1923 0455 1936 0513 1924 0539 1849 0604 1802 0631 1721 0703 1657 0724 1702

27 0719 1732 0647 1804 0605 1831 0522 1858 0456 1923 0455 1936 0514 1923 0540 1847 0605 1801 0632 1720 0704 1657 0725 1703

28 0718 1733 0645 1805 0603 1832 0521 1859 0456 1924 0455 1936 0515 1923 0540 1846 0606 1759 0633 1719 0705 1656 0725 1704

29 0718 1734 0644 1806 0602 1833 0520 1900 0455 1925 0456 1936 0516 1922 0541 1844 0606 1758 0634 1718 0706 1656 0725 1704

30 0717 1735			 0600 1834 0519 1901 0455 1926 0456 1936 0516 1921 0542 1843 0607 1756 0635 1717 0707 1656 0725 1705

31 0716 1736			 0559 1834			 0454 1926			 0517 1920 0543 1842			 0636 1715			 0726 1706

</pre>

<p><a href="javascript:history.go(-1)">Back to form</a></p>

</body>

</html>

while this looks like it it doesn't line up pasting it into TextMate and looking at web preview it seems fine



with this code, which looks the same as yours, although simplified a bit

// PostDataToURL ( key ; value ; url ; dataKeys ; dataValues )

// 12_10_13

// v1.1

//

data = 'key' + '=' + 'value'

i = 0

if (dataKeys){

dk = dataKeys.tokenize('n')

dv = dataValues.tokenize('n')

dk.each(){

data += '&' + it +'=' + dv[i]

i++

}

}

try{

// Send data

url = new URL(url)

conn = url.openConnection()

conn.setDoOutput(true)

wr = new OutputStreamWriter(conn.getOutputStream())

wr.write(data)

wr.flush()

// Get the response

response = conn.getInputStream().getText('utf-8')

wr.close()

} catch (e) {

return e

} //end try

return response

And interestingly you don't seem to need the FFX and2 either!!

[john renfrew]

If you add this to the bottom it will return just the data as CSV for you..

//return response

res = response[response.lastIndexOf('h m')+4..-1]

res = res[1..res.indexOf('</pre>')-1].trim()

res = res.replace('			 ', 'ttt')

res = res.replace(' ', 't').replace(' ', 't')

return res

bit clunky and whizz with Regex could do a much better job

[xjonnie]

Thanks! Im chewing on this, Doesnt compile at first glance.

My eventual goal is to parse each date on the table to a field/record.

Have an idea to read the page and extract the inputs.

An automated way to add a page without going through the source manually.

Something anyone one could setup.

[xjonnie]

Thanks Again!

Have any code for a fixed field??

Some of the tables can actually have a blank for the date or spaces actually??

[JerrySalem]

xjonnie,

why didn't you hard code the key name?

will they change?

 

Jerry