Comments on: Creating Web Scraping Tools for MSPs with PowerShell https://www.altaro.com/msp-dojo/web-scraping-tool-for-msps/ Managed Service Provider guides, how-tos, tips, and expert advice Wed, 24 Mar 2021 11:36:03 +0000 hourly 1 By: Luke Orellana https://www.altaro.com/msp-dojo/web-scraping-tool-for-msps/#comment-1032 Mon, 02 Sep 2019 16:51:56 +0000 https://www.altaro.com/msp-dojo/?p=1224#comment-1032 In reply to Andrew.

Awesome suggestion! I just tested this and since the data is so small I didn’t get that much of a decrease in time. However, I’m definitely going to keep this design pattern in my back pocket for dealing with larger data sets. Thank you!!!

]]>
By: Andrew https://www.altaro.com/msp-dojo/web-scraping-tool-for-msps/#comment-1031 Mon, 12 Aug 2019 10:32:43 +0000 https://www.altaro.com/msp-dojo/?p=1224#comment-1031 Hi, I noticed that in your script that you use new-object, add-member, and add to the table using +=.
These are all costly cmdlets in terms of speed and memory use, particularly the += operation.

You can speed up the script by 2-3 seconds /every loop/ by making these small changes:

Create the table outside of the for loop ahead of time as an array list:
[system.collections.arraylist]$table = @()

You can also create your object in one go using hash table notation, rather than 3 seperate cmdlet calls:
$row = [psobject]@{
Sale = $sale;
Link = $links[$index]
}

Then you can add to the array list using a method, instead of the insanely costly += operation:
$table.add($row) | out-null

I hope you find this useful!

]]>