Computer Hope
Software => Computer programming => Topic started by: Santosh on August 14, 2020, 03:21:16 AM
-
Below is the Batch script from which I am checking the URLs status by CURL statement, but it is redirecting all output to NotOKURL.txt file.
I want to print only those URLs which are not working in the NotOKURL.txt file. Please help
:Check_URL
@for /F "usebackq delims=" %%u in ("%URL%") do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && (
echo "%%u" ==^> is OK
) || (
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
-
This may help:
:Check_URL
for /f %%u in ("%url%") do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
Not certain why you need that last curl statement.
Happy Coding. 8)
-
Thanks for the code sir but it is not working.
I am expecting the code should check the URLs status from the text file and send the output only to Not_OK_URL.txt file if any URL is not working.
And send the mail to the receipients that Not_OK_URL.txt file as a attachment.
URL.txt:
https://gorest.co.in/public-api/users
https://gorest.co.in/public-api/posts
https://gorest.co.in/public-api/comments
https://gorest.co.in/public-api/albums
:Check_URL
for /f %%u in ("%url%") do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
If Exist "%Not_OK_URL%" blat %Sender% -subject "URLs DOWN STATUS" -body "please check the attachment for the Application DOWN error logs" %SMTP_Server% -attach C:\Not_OK_URL.txt %Log% -to [email protected]
pause
Your quick response will be appreciated.
-
Hi,
Could you please help me to resolve this.
Your quick response will be appreciated.
-
I missed the quotes around the url variable in the for statement. They should not be there.
:Check_URL
for /f %%u in (%url%) do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
If Exist "%Not_OK_URL%" blat %Sender% -subject "URLs DOWN STATUS" -body "please check the attachment for the Application DOWN error logs" %SMTP_Server% -attach C:\Not_OK_URL.txt %Log% -to [email protected]
pause
I do not have blat on my machine. If blat fails, maybe someone else can help. The NOT_OK_URL file logic should be OK now.
8)
-
Yes I have tried with the above code which you have given but still after the execution of the code if I check the Not_OK_URL.txt file it is printing even for OK urls also. I mean which are connecting and working fine, it is printing that also. Instead I need like it should only print when the URL is not working and connecting.
Below are the sample output from Not_OK_URL.txt after executing the present code.
---------------------------------------------------
"https://gorest.co.in/public-api/users"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 18 Aug 2020 06:02:37 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"ae5573f6afa9a1e5e810abe0e24264a7"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: e1cd7e0b-015a-4898-ac11-4b6ba387b387
X-Runtime: 0.005474
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
---------------------------------------------------
"https://gorest.co.in/public-api/posts"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 18 Aug 2020 06:02:39 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"6b6c39b4c265c16d413c1147f1bfb2cd"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: bf8c5f06-d397-4cc3-8458-47f3d695c8f9
X-Runtime: 0.007588
-
Odd situation. I ran my posted code using the URL.txt file you posted with the following results:
G:\WFC\Testlib>type Not_OK_URL.txt
---------------------------------------------------
"https://gorest.co.in/public-api/albums"
---------------------------------------------------
HTTP/1.1 404 Not Found
Server: nginx
Date: Tue, 18 Aug 2020 11:26:33 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
X-Request-Id: 74e442bf-33ea-4d62-9da4-7111844d0381
X-Runtime: 0.005282
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
I also checked the URL.txt file contents manually, and the results were that the first three urls are valid but not the fourth, which matched the results found in the Not_OK-URL.txt file.
How and what values are assigned to the %url% and %Not_OK_URL% variables? How is control passed to the :Check_URL label?
8)
The script you initially posted was missing a pair of parenthesis that were needed to send block output to the NOT_OK_URL file. It might be helpful if you post the entire batch file (if it's not too big).
-
Yes the code is working fine now. But I need to send a mail notification with that Not_OK_URL.txt file as attachement in the mail to receipients.
How I can achieve this. Could you please help me.
-
Why not continue to use BLAT as described in your first post? As I mentioned, I have no experience with this utility and couldn't find much documentation on the web, but this may help (http://www.blat.net/). Check the links in the doc files section in the left column.
8)
-
The above script is working fine. But I want Not_OK_URL.txt file should be stored in root path, I mean whatever the path we mention, it should store there so that we can use it as an attachment by mentioning that path and access it easily.
Could you please help me to alter this script.
:Check_URL
for /f %%u in (%url%) do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
If Exist "%Not_OK_URL%" blat %Sender% -subject "URLs DOWN STATUS" -body "please check the attachment for the Application DOWN error logs" %SMTP_Server% -attach C:\Not_OK_URL.txt %Log% -to [email protected]
pause
-
And one more thing, can we write the same script in Windows powershell? If so can you please help me with that.
:Check_URL
for /f %%u in (%url%) do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
If Exist "%Not_OK_URL%" blat %Sender% -subject "URLs DOWN STATUS" -body "please check the attachment for the Application DOWN error logs" %SMTP_Server% -attach C:\Not_OK_URL.txt %Log% -to [email protected]
pause
-
I came up with this. If you want files in a certain folder, change the first two lines accordingly.
$url = "url.txt"
$not_ok_url = "not_ok_url.txt"
Get-Content $url | foreach {
$site = $_
try {
$response = Invoke-WebRequest $_
Write-Output "$site OK"
} catch {
Write-Output "$site NOK"
("-" * 75) | Out-File -FilePath $not_ok_url -Append -encoding ascii
$site | Out-File -FilePath $not_ok_url -Append -encoding ascii
("-" * 75) | Out-File -FilePath $not_ok_url -Append -encoding ascii
$_.Exception | Out-File -FilePath $not_ok_url -Append -encoding ascii
}
}
if ( Test-Path($not_ok_url) ) {
Invoke-Expression -Command "cmd.exe blat.bat"
}
This will run in both Powershell .Net Framework (5.1) and Powershell .Net Core (7.3). Now the bad news. The Send-MailMessage cmdlet no longer works in either Powershell version. You can use the MailKit module in Powershell Core but it's not pretty and I only got it to work with Gmail. Instead I suggest you write a blat.bat file to set your variables and run blat as usual in a cmd environment. I set that up in the Powershell script, but you'll have to write the code for the blat batch file.
Hope this helps 8)
PS. Curl is an alias for Invoke-Webaddress.
-
Yes this script works fine, but it is just printing the Site is NOK.
But I want to check the output as
---------------------------------------------------
"https://gorest.co.in/public-api/users"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 18 Aug 2020 06:02:37 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"ae5573f6afa9a1e5e810abe0e24264a7"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: e1cd7e0b-015a-4898-ac11-4b6ba387b387
X-Runtime: 0.005474
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
Right now it is not giving the Site information like which service is down of Site.
-
The below code is not giving me the correct output.
Expected: Not_OK_URL.txt file should only contain the URL with its status message of not working URL. But it is pasting everything, I mean it is printing even OK URL status also.
Please help here to print only not working URL status.
Set "URL=%~dp0URL.txt"
set Log=-log "%LOG_BLAT%"
Set "Not_OK_URL=%~dp0Not_OK_URL.txt"
If Exist "%Not_OK_URL%" Del "%Not_OK_URL%"
:Check_URL
for /f %%u in (%url%) do (
curl -I -s "%%u" | find /I "200">nul 2>&1 && echo "%%u" ==^> is OK || (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
Result:
---------------------------------------------------
"https://gorest.co.in/public-api/users"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 03 Sep 2020 14:03:17 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"f67409a2a3c272242a3cebf9a85341f2"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 22e65e92-334b-40a7-a307-26554fa186ad
X-Runtime: 0.004949
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
Expect-CT: enforce; max-age=3600
Feature-Policy: geolocation none;midi none;notifications none;push none;sync-xhr none;microphone none;camera none;magnetometer none;gyroscope none;speaker self;vibrate none;fullscreen self;payment none;
---------------------------------------------------
"https://gorest.co.in/public-api/posts"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 03 Sep 2020 14:03:19 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"7561a5abdfd1e1e76a057bb0f5103077"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: c20e408b-d6c4-4411-b45e-080c71644fc6
X-Runtime: 0.005973
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
Expect-CT: enforce; max-age=3600
Feature-Policy: geolocation none;midi none;notifications none;push none;sync-xhr none;microphone none;camera none;magnetometer none;gyroscope none;speaker self;vibrate none;fullscreen self;payment none;
---------------------------------------------------
"https://gorest.co.in/public-api/comments"
---------------------------------------------------
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 03 Sep 2020 14:03:21 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"764d697d71c4dced138a6b658c4fdcb6"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 0eed26bd-511e-43d8-b97d-6e8089681d0d
X-Runtime: 0.004440
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
Expect-CT: enforce; max-age=3600
Feature-Policy: geolocation none;midi none;notifications none;push none;sync-xhr none;microphone none;camera none;magnetometer none;gyroscope none;speaker self;vibrate none;fullscreen self;payment none;
---------------------------------------------------
"https://gorest.co.in/public-api/albums"
---------------------------------------------------
HTTP/1.1 404 Not Found
Server: nginx
Date: Thu, 03 Sep 2020 14:03:24 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
Cache-Control: no-cache
Set-Cookie: _gorest_session=%2FAM6FbTGAcov%2BR%2B2YA710CDaXn%2BDgHIyAIM4IeBb2Lv%2F4JCncjxWbDz1L9vIIwX%2Bdw8qi7%2F7I07zpjmkuLCIST4aLpdgKTh%2FXQxWZW%2BupfvlKjBkANDGQ9gbfDmHM6rarNSCrqTNXHm1uyVkSqfrYnGJbHvIEVbvgxqeKXLKcKyA1%2Fqe8ysUwMTt1EZsmAhikscIHqFNAziur8JD7R7EWtybVcZKNAGzTSwMQxHPWa8oI%2FPhwNkuaY8XbUwZgSTcMAiHJkopV8k01iEbrTXdpng%2BHS5p0Bc%3D--sktyCmRyx6aCiXOy--QYgw0tsiIFobS3LXDMAxvw%3D%3D; path=/; secure; HttpOnly
X-Request-Id: e926309e-be24-4776-ab8d-c0c30dac7002
X-Runtime: 0.004771
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Origin
X-Frame-Options: sameorigin
-
My last post contained a few misstatements:
The current release of Powershell Core is 7.0.3 not 7.3
Mailkit is not a Powershell module; it is a Nuget package installed with the Install-Package cmdlet
Curl is not an alias for Invoke-Webrequest; they are functionally similar but not interchangeable.
Are you going back to the batch solution? If so, this may help:
@echo off
setlocal enabledelayedexpansion
set url="url.txt"
set Not_OK_Url="Not_OK_Url.txt"
if exist %Not_OK_Url% del %Not_OK_Url%
for /f "usebackq delims=" %%u in (%url%) do (
curl -I -s "%%u" | find /I "200">nul 2>&1 ^
&& echo "%%u" ==^> is OK ^
|| (
(
echo ---------------------------------------------------
echo "%%u"
echo ---------------------------------------------------
curl -I -s "%%u"
)>>"%Not_OK_URL%"
)
)
:: If Exist "%Not_OK_URL%" blat %Sender% -subject "URLs DOWN STATUS" -body "please check the attachment for the Application DOWN error logs" %SMTP_Server% -attach C:\Not_OK_URL.txt %Log% -to [email protected]
I think you're better off with batch [never thought I'd say that]. If you go with Powershell, I strongly suggest Powershell Core (Powershell Windows is no longer in production), but you also have to consider that the cmdlets may not give you all the data you need for this project.
Best Of Luck 8)
-
This script runs under Powershell Windows, uses Curl and I made an attempt at a Powershell version of the Blat command line:
$url = "url.txt"
$not_ok_url = "not_ok_url.txt"
$work = "response.txt"
[regex] $regex = @"
(?smi)(.*200\sOK)
"@
if ( Test-Path $not_ok_url ) { Remove-Item $not_ok_url }
Get-Content $url | foreach {
$site = $_
Start-Process -FilePath "curl.exe" -ArgumentList "-I -s $_" -NoNewWindow -RedirectStandardOutput $work -Wait
$response = Get-Content $work -Raw
if ( $response -match $regex) {
Write-Output "$site ... OK"
} else {
Write-Output "$site ... NOK"
("-" * 75) | Out-File -FilePath $not_ok_url -Append -encoding ascii
$site | Out-File -FilePath $not_ok_url -Append -encoding ascii
("-" * 75) | Out-File -FilePath $not_ok_url -Append -encoding ascii
$response | Out-File -FilePath $not_ok_url -Append -encoding ascii
}
if ( Test-Path $work ) { Remove-Item $work }
}
# if ( Test-Path($not_ok_url) ) {
# $sender =
# $log =
# $smtp =
# Start-Process -FilePath "blat.exe" -ArgumentList "$sender -subject 'URLs DOWN STATUS' -body 'please check the attachment for the Application DOWN error logs' $smtp -attach 'C:\Not_OK_URL.txt' $log -to '[email protected]'
# }
You can tweak the paths at the top of the script. The Blat section is commented out but if remove the comment marks (#) and fill in the values for the variables, it just might work.
8)