I would like to get one of those Raspberry Pi Model 4 with at least 4 gig of RAM. They are really kind of hard to get a hold of right now. I know there is that web site out there that you can monitor the Raspberry Pi retailers, but I don’t want to have to check a web site every hour hoping to catch one for sale. I wanted to also have a way of monitoring them that uses little cpu and small amount of real estate on my desktop monitor.
Here is my plan, to write a small GUI application that checks for availability every few minutes and display the results in a small table format. First question is how to check for availability. Well, I could wget each website into a sub-directory and then scan the files for a key word.
For each retailer (I am only looking at US Retailers right now) the web site will be different and for each web site the key word may also be different. I also discovered that I was going to need a count for how may times I was going to find that key word in the HTML source. This has Array written all over it. An Array for the web sites, an array for the key words and finally an Array for the counts.
For those who don’t know what an Array is, one my text books describes an Array as Homogeneous data structure, basically that means similar data items are stored under one name. So, basically it is a way to group similar types of data together. It is also the most common data structure in data processing. Each separate item can then be accessed by the name with an index. Each index is a whole number (Integer) which cannot exceed the size of the array. If you exceed the size or use a negative number, the program will usually crash. Arrays can usually be of one simple type of item or a more complex structure.
I did this in Lazarus, which is a GUI front end to the Free Pascal Language. It is a Form based system that is friendly and very easy to work with. It is also Object Oriented with many visual components. The main ones I will be using here are the Tbutton and TStringGrid. The Tbutton allows the user to start the whole process by just clicking on the button, the TstringGrid allows a structured and pretty way of displaying the info.
Lets look at the Arrays first. I could have made this one big complex structure, but since they were going to be mostly constants and I thought it would be easier to look at separate simpler basic arrays. Free Pascal is a superset of Turbo Pascal from the early 1980s and then in the late 1980-1990s object oriented data structures were added. Niklaus Wirth’s Pascal from the 1970s didn’t have a string library built in, but Turbo Pascal did and it made a much more usable programming language. Here, you will see that I have used String type in a number of my Arrays.
Const
StoreLabel: Array[1..20] of String = ('AdaFruit', 'CanaKit-1', 'CanaKit-2',
'CanaKit-4', 'CanaKit-8','PiShop-1', 'PiShop-2','PiShop-4',
'PiShop-8', 'Okco-2', 'Okco-4', 'Okco-8', 'Chicago-1gb',
'Chicago-2gb', 'Chicago-4gb', 'Chicago-8gb','SparkFun-2',
'SparkFun-4', 'SparkFun-8', 'MicroCenter');
WebAddr: Array[1..20] of String = ('https://www.adafruit.com/product/4295',
'https://www.canakit.com/raspberry-pi-4.html',
'https://www.canakit.com/raspberry-pi-4-2gb.html',
'https://www.canakit.com/raspberry-pi-4-4gb.html',
'https://www.canakit.com/raspberry-pi-4-8gb.html',
'https://www.pishop.us/product/raspberry-pi-4-model-b-1gb/',
'https://www.pishop.us/product/raspberry-pi-4-model-b-2gb/',
'https://www.pishop.us/product/raspberry-pi-4-model-b-4gb/',
'https://www.pishop.us/product/raspberry-pi-4-model-b-8gb/',
'https://www.okdo.com/us/p/raspberry-pi-4-model-b-2gb/',
'https://www.okdo.com/us/p/raspberry-pi-4-model-b-4gb/',
'https://www.okdo.com/us/p/raspberry-pi-4-model-b-8gb/',
'https://chicagodist.com/products/raspberry-pi-4-model-b-1gb?src=raspberrypi',
'https://chicagodist.com/products/raspberry-pi-4-model-b-2gb?src=raspberrypi',
'https://chicagodist.com/products/raspberry-pi-4-model-b-4gb?src=raspberrypi',
'https://chicagodist.com/products/raspberry-pi-4-model-b-8gb?src=raspberrypi',
'https://www.sparkfun.com/products/15446',
'https://www.sparkfun.com/products/15447',
'https://www.sparkfun.com/products/16811',
'https://www.microcenter.com/product/609038/4--model-b-4gb?src=raspberrypi');
SearchFor: Array[1..20] of String = ('Out of stock', 'Sold Out',
'Sold Out', 'Sold Out', 'Sold Out', 'Out of stock',
'Out of stock', 'Out of stock', 'Out of stock',
'Out of stock', 'Out of stock', 'Out of stock', 'Sold Out',
'Sold Out', 'Sold Out', 'Sold Out',
'do not currently have an estimate',
'do not currently have an estimate',
'do not currently have an estimate',
'UNAVAILABLE ONLINE');
SearchCnt: Array[1..20] of LongInt = (12, 4, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3,
1, 1, 1, 1, 2, 2, 2, 1);
The source code looks like the following:
1 Procedure BuildHeading(Var SG: TStringGrid);
2 Begin
3 SG.Clear;
4 SG.ColCount := 5;
5 SG.RowCount := 30;
6 SG.Cells[0, 0] := 'Store Name';
7 SG.Cells[1, 0] := 'How Many';
8 SG.Cells[1, 1] := 'Should Be';
9 SG.Cells[2, 0] := 'How Many';
10 SG.Cells[2, 1] := 'There are';
11 SG.Cells[3, 0] := 'Differance';
12 SG.Cells[4, 0] := 'Status';
13
14 End;
15
16 Procedure DeleteTempFiles();
17 Var
18 fn: String;
19 Info: TSearchRec;
20 Begin
21 if FindFirst(KkDirStr + '/*.*',faAnyFile,Info) = 0 Then
22 Repeat
23 fn := Info.Name;
24 If ((fn <> '.') and (fn <> '..')) Then
25 DeleteFile(KkDirStr + '/' + fn);
26 until FindNext(info)<>0;
27 FindClose(Info);
28 end;
29
30 Procedure StartWGet(Addr: String);
31 Var
32 InternetProcess: TProcess;
33 Begin
34 InternetProcess := TProcess.Create(nil);
35 InternetProcess.Executable:= 'wget';
36 InternetProcess.Parameters.Add('--directory-prefix=' + KkDirStr);
37 InternetProcess.Parameters.Add(Addr);
38 InternetProcess.Options := InternetProcess.Options + [poWaitOnExit];
39 InternetProcess.Execute;
40 InternetProcess.Free;
41 end;
42
43 Function FindKeywordCnt(KeyStr: String): LongInt;
44 Var
45 KeyWordCnt, i, j: LongInt;
46 fn, WorkStr: String;
47 Info: TSearchRec;
48 F: TextFile;
49 StrSearched: Boolean;
50 Begin
51 KeyWordCnt := 0;
52 FindFirst(KkDirStr + '/*.*',faAnyFile,Info);
53 Repeat
54 fn := Info.Name;
55 If ((fn <> '.') and (fn <> '..') and (Length(fn) > 2)) Then
56 Begin
57 AssignFile(F, KkDirStr + '/' + fn);
58 Reset(F);
59 ReadLn(F, WorkStr);
60 Repeat
61 i := Length(KeyStr);
62 StrSearched := False;
63 For j := 1 To Length(WorkStr) Do
64 Begin
65 If Copy(WorkStr,j,i) = KeyStr Then
66 KeyWordCnt := KeyWordCnt + 1;
67 end;
68 ReadLn(F, WorkStr);
69 Until Eof(F);
70 CloseFile(F);
71 end;
72 Until FindNext(info)<>0;
73 FindClose(Info);
74 FindKeyWordCnt := KeyWordCnt;
75 end;
76
77 Procedure DisplayDtlLine(Var SG: TStringGrid; RowVal: LongInt;
78 StoreLabel: String; Fnd, ExpCnt: LongInt; Var TimeVar: TDateTime);
79 Const
80 MyEmailAddr = 'MyEmailAddr';
81 MyGMailAddr = 'MyGMailAddr';
82 MyEmailSvr = 'MyEmailSvr';
83 MyEmailUser = 'MyEmailUsr';
84 MyEmailPass = 'MyEmailPass';
85 Var
86 Msg: TStringList;
87 Begin
88 Msg := TStringList.Create;
89 SG.Cells[0, RowVal+2] := StoreLabel;
90 SG.Cells[1, RowVal+2] := IntToStr(ExpCnt);
91 SG.Cells[2, RowVal+2] := IntToStr(Fnd);
92 SG.Cells[3, RowVal+2] := IntToStr(Fnd-ExpCnt);
93 If Fnd < ExpCnt Then
94 Begin
95 SG.Cells[4, RowVal+2] := 'Available';
96 If Now() > IncHour(TimeVar, 3) Then
97 Begin
98 Msg.Clear;
99 Msg.Add(StoreLabel + ' May have RaspPi''s');
100 SendToEx(MyEmailAddr, MyGmailAddr,
101 StoreLabel + ' May have RaspPi''s', MyEmailSvr,
102 Msg, MyEmailUser, MyEmailPass);
103 TimeVar := Now();
104 end;
105 end
106 Else SG.Cells[4, RowVal+2] := 'UnAvailable';
107 Msg.Free;
108 End;
109
110 procedure TMainForm.BeginButClick(Sender: TObject);
111 var
112 i, j, k, P, CntFnd: LongInt;
113 Done, StrSearched: Boolean;
114 WorkStr: AnsiString;
115 Info: TSearchRec;
116 fn: String;
117 EndTime: TDateTime;
118 TimeArray: Array[1..20] of TDateTime;
119 begin
120 Done := False;
121 For i := 1 To 20 Do
122 TimeArray[i] := Now();
123 Repeat
124 BuildHeading(RaspPiSG);
125 For i := 1 To 20 Do
126 Begin
127 DeleteTempFiles();
128 StartWGet(WebAddr[i]);
129 CntFnd := FindKeyWordCnt(SearchFor[i]);
130 DisplayDtlLine(RaspPiSG, i, StoreLabel[i], CntFnd, SearchCnt[i],
131 TimeArray[i]);
132 end;
133 RaspPiSG.Cells[0, 23] := 'Last Refresh';
134 RaspPiSG.Cells[1, 23] := TimeToStr(Now());
135 RaspPiSG.AutoSizeColumns;
136 RaspPiSG.Repaint;
137 EndTime := Now + EncodeTime(0, 0, 45, 0); // 45 Seconds
138 While Now < EndTime do
139 Begin
140 Application.ProcessMessages;
141 Sleep(10);
142 end;
143 until Done;
144
145 end;
Lets step through the Procedure and talk about how it works. The TButton Procedure that Lazarus created is clear down on line 110. You can see the actual Procedure is fairly short because it calls sub procedures and functions to do most of the work.
“Done” is first thing set in this procedure, it is a boolean variable which means it can be set to “True” or “False” only. I have set it to “False” and will execute a “Repeat” until Done equals “True”, that will never happen and this procedure will run for ever or at least until I kill the program.
The "For" loop sets an array of TdateTime to the current time(lines 121-122). I want to send myself an email if a Pi becomes available.
Wait … Did I say email???
Well, it’s too easy to miss it on the screen and a message to my phone is a another way to notify that some Pi’s have become available. But I don’t want to send that email every minute or even multiple times per hour, so I am keeping a list of the last time I sent an email and I preset last time email was sent to the startup of the process.
On line 123 we have the Repeat, the Until Done is clear down on line 143. This loop should never end. The first thing I do in the "repeat loop" is call the “BuildHeading Procedure. The “BuildHeading” Procedure is on lines 1-14. The most interesting thing about that procedure is that I pass the RaspPiSG as a Var which allows me to modify it in the called procedure. Note in the called procedure I just refer to it as SG. The names of variables do not have to be the same between the calling and the called procedures, but the types and order of called variables have to align.
On lines 125-132, I execute a for loop, inside this loop I call several procedures and a function.
On Line 128 I call the Procedure “DeleteTempFiles” which is on lines 16-28. Every time we run the Linux program “wget” we download files from a webpage. That directory needs cleaned out before downloading the next retailers website, so I execute a “FindFirst”...”FindNext” and then delete every file it finds. Many languages use similar functions to accomplish something like this.
On line 128, I call the “StartWGet” Procedure (lines 30-41). And that is exactly what it does. It runs “wget” as a background process and control of my program is blocked until “wget” finishes. “wget” dumps the HTML source files into the work directory. This uses a Tprocess data structure defined in the Var section. It also requires “Process be declared in the Uses deceleration which pulls in the correct libraries. “wget” is a standard Linux program used to grab HTML files from websites, many distros have it installed by default. If not, “wget” should be in your distro’s repository. Using Tprocess to execute an external process is explained in the following website:
https://wiki.freepascal.org/Executing_External_Programs
Note the variable we pass to “StartWGet”
WebAddr[i]
is referring to one of the array elements where “i” is an integer. The integer value in “i” combined with the array name refers to a unique element in that array. Using the name “i” for the index for element of an array is kind of historic and goes back to the early days of Fortran.
On line 129, I call the FindKeyWordCnt Function (Lines 43-75). This Function also does a “FindFirst”...”FindNext” loop, but instead of deleting the files, it counts how many times the KeyWord for that specific Retailer is in the HTML files. Finally the Integer is returned to the CntFnd variable.
Finally on line 130-131, I call the DisplayDtlLine Procedure(Line 77-108) which displays the current detail line in our TstringGrid. Note I passed the TstringGrid in this procedure just like I did in the BuildHeading Procedure. In this procedure is where I send an email to my GMail Account. Msg is defined in this Procedure as a TstringList, which is an predefined object defined in the “Classes” library. TstringList is a sort of structured array of strings with some programming logic added. A TStringList is required to send an email through the app, the TStringList contains the email body’s content. If you check out Lazarus, you will find many visual components use TstringList as a Foundational Element. Listboxes, dropdown-boxes and database components all use the TstringList internally.
This is where my for loop ends and we bounce back to finishing the Repeat Statement.
Lines 133-136 causes my TStringGrid to display a little nicer. The columns get sized correctly and the time of last display is printed.
Performance: It doesn’t seem that bad. Watching it through Top, the program never pulls more than a 1% or 2% on one cpu. The only bottleneck is running all of the wget’s in sequence. I could set it up to run the wget’s in batches. If I ran five at a time it would finish executing the wget’s much quicker. It would make the program a little more complex and requires multiple work directories, but it would be doable.
Email: The procedure I use to send email comes from Synapse’s website. You have to download their source-code and add it as libraries to your app. It is totally procedural. http://www.ararat.cz/synapse/doku.php/start
I would like to consider something other than email, GMail is blocking some of my emails. I assume they think I am doing something impoper. I looked at sending IM’s, but that would cost money and I like free. My Wife suggested I check out Signal, haven’t gotten around to that yet. I would need a good command line method to send the messages and to easily receive them on my phone. The email is actually sent on lines 100-102. I first check to see if an email has been sent recently, if not a new one will be sent. Once a new email is sent, that Retailer’s TimeArray element is updated with the current Date/Time.
Results: I haven’t
got a Pi yet. Came close a couple of times. I wrote an additional
program that lets me select the site and it pulls up the website. I
can get to each website in just a few seconds. The additional
program was required because when I put the program to sleep, it
makes the program unresponsive. While the second program is very
responsive. Microcenter has become a problem. They changed their
website and now I haven’t found a good keyword, the keyword that is
displayed on the website doesn’t show up in HTML source code that
is downloaded. A number of the other websites has changed and I have
had to change a few keywords to get good results. This is an image of my program running:
