It is currently April 27th, 2024, 5:21 pm

Market Prices

RSS, ATOM and other feeds, GMail, Stocks, any information retrieved from the internet
User avatar
Yincognito
Rainmeter Sage
Posts: 7178
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Market Prices

Post by Yincognito »

Mordasius wrote: February 16th, 2024, 10:02 pmI'm also perplexed by this but can now confirm that the downloaded files (see attached) are showing the correct Price Changes but sometimes not updating the Price accordingly. I got up very early (for me) to catch the relevant US stocks whilst the markets were still open and noted the following - check the Price and Price Changes for NASDAQ and Home Depot as examples.
Yincognito wrote: February 16th, 2024, 11:04 pm If this is true for all cases (and not just for Home Depot or this particular instance when the market was closed)
Well, I took a brief look at the "dump" files you posted, and they seem to have more than one instance of the <last> field, just like what I posted above. So far, grabbing the proper <last> aka "price" seems like the obvious "fix" for this - if, of course, the <last> value that is not currently extracted by the skin is actually the correct one (which I suppose it is, based on my test earlier). In any case, it will be your call / confirmation / tests to see if it's true, as it's your skin and all. ;-)
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: Market Prices

Post by Mordasius »

Yincognito wrote: February 16th, 2024, 11:04 pm Now, I don't know if I queried this at the right time or not, or whether it's the same for other stocks, but in this case it looks to me that the problem is not that the .xml contains the wrong data, but the fact that the skin extracts the "other" / "wrong" instance of the <last>...</last> field (and yes, there is in fact a <last>...</last> field in the .xml that has the "correct" 362.35 value). Or, it's the other way around and the "wrong" instances of <change>...</change> and <change_pct>...</change_pct> are extracted. If this is true for all cases (and not just for Home Depot or this particular instance when the market was closed), it can be corrected by minor adjustments to the RegExp options, which I intentionally let very simple and easy to adjust if needed.

I guess some more tests will be needed on this, before jumping to a hasty decision - just my opinion..
Yes the second <last>...</last> field gives the correct current price in most cases but unfortunately not all. The Dow Jones, S&P 500 and Travelers Companies only have a single <last>...</last> field.

Does anyone know a RegExp to get the second <last>...</last> field if there is one if not return the first <last>...</last> captured?

If not, I'm going to use the <previous_day_closing>...</previous_day_closing> plus the <change>...</change> to give what I hope will be the correct current price.
ComputedPrices.png
The column on the right is the computed price. The computed prices are the same as those in the second <last>...</last> field except for the Dow Jones, S&P 500 and Travelers Companies where the price in the first <last>...</last> field is correct. I'll have to wait until the markets open on Monday so that I can test it with some real data but I think it should work.
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Rainmeter Sage
Posts: 7178
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Market Prices

Post by Yincognito »

Mordasius wrote: February 17th, 2024, 10:24 am Yes the second <last>...</last> field gives the correct current price in most cases but unfortunately not all. The Dow Jones, S&P 500 and Travelers Companies only have a single <last>...</last> field.

Does anyone know a RegExp to get the second <last>...</last> field if there is one if not return the first <last>...</last> captured?
Didn't test it, but you can try some of the following:

Code: Select all

(?siU).*?<last>(.*)<\/last>
(?siU)^.*?<last>(.*)<\/last>
(?siU)^.*?<last>(.*)<\/last>.*$
Basically, since the (?U) ungreedy flag is set at the start, you'll need to "invert" it by adding a ? after the first * to make .*? match as many chars as possible before <last>, rather than as few as possible (like .* matches because of the overall ungreedy setting at the start). The effect is that the last <last> will be captured - see what I did here? :D Obviously, if there's a single <last>, then that will be captured. So yep, a single ? in the right place should fix this - of course, it will remain to be seen if the approach works with real data in your further tests.

Generally, it isn't really about whether the <last> you capture is the first or the last, but about which <last> corresponds to the data you want displayed (after all, that's why you have <subsections>...</subsections> in the <quickQuote> section). A proper approach would be related to such sections / subsections rather than the order in the string, but that obviously requires extensive knowledge of what such financial data means, as well as of the entire structure of such a query response from CNBC. Even so, that is volatile according to the stock type and such, and it would complicate such a section based matching a bit.

The important thing is that the correct data seems to be present in the query response. The rest is about picking the right occurence and can be controlled by the skin designer / user anyway.
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: Market Prices

Post by Mordasius »

Yincognito wrote: February 17th, 2024, 3:43 pm Didn't test it, but you can try some of the following:

Code: Select all

(?siU).*?<last>(.*)<\/last>
(?siU)^.*?<last>(.*)<\/last>
(?siU)^.*?<last>(.*)<\/last>.*$
Basically, since the (?U) ungreedy flag is set at the start, you'll need to "invert" it by adding a ? after the first * to make .*? match as many chars as possible before <last>, rather than as few as possible (like .* matches because of the overall ungreedy setting at the start). The effect is that the last <last> will be captured - see what I did here? :D Obviously, if there's a single <last>, then that will be captured. So yep, a single ? in the right place should fix this - of course, it will remain to be seen if the approach works with real data in your further tests.
We there you go all it takes is .*? to invert the ungreedyness. I would never have come up with that but it seems that all three of your candidate RegExps grab the second <last>..<\/last> number if it is there and if not return the first <last>..<\/last> number. Many Thanks that's exactly what is needed.

I've dropped this option into a skin with the original Price along with the computed price ( <previous_day_closing> plus the <change> ) and the price returned by your RegExps ready for proper testing when the markets open on Monday.
thrreshots.png
You do not have the required permissions to view the files attached to this post.
User avatar
Yincognito
Rainmeter Sage
Posts: 7178
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Market Prices

Post by Yincognito »

Mordasius wrote: February 18th, 2024, 7:11 am We there you go all it takes is .*? to invert the ungreedyness. I would never have come up with that but it seems that all three of your candidate RegExps grab the second <last>..<\/last> number if it is there and if not return the first <last>..<\/last> number. Many Thanks that's exactly what is needed.

I've dropped this option into a skin with the original Price along with the computed price ( <previous_day_closing> plus the <change> ) and the price returned by your RegExps ready for proper testing when the markets open on Monday.

thrreshots.png
Excellent! If by any chance you want to also avoid potential errors that could "panic" users if fields are not found (e.g. if the quickQuote of a stock is empty for whatever reason), you might want to use ((?=...)...) lookahead assertions in the WebParser child measures as well. It's not mandatory though.
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
sk14
Posts: 25
Joined: February 9th, 2024, 7:21 pm

Re: Market Prices

Post by sk14 »

Just a heads up, the NYSE will be closed on Monday due to Presidents day. I'm glad that I wasn't crazy about the price issues with the skin. Sounds like it is very close to being solved. Thanks for all your hard work on one of the best skins for Rainmeter.

Steve
User avatar
Yincognito
Rainmeter Sage
Posts: 7178
Joined: February 27th, 2015, 2:38 pm
Location: Terra Yincognita

Re: Market Prices

Post by Yincognito »

sk14 wrote: February 18th, 2024, 5:47 pm I'm glad that I wasn't crazy about the price issues with the skin. Sounds like it is very close to being solved.
You certainly weren't, it's just that proof and ways to tackle it were needed via examining the .xml response from the site. Hopefully the fix will be efficient for all cases, that will be up to Mordasius to see during his tests with real-time data. ;-)
Profiles: Rainmeter ProfileDeviantArt ProfileSuites: MYiniMeterSkins: Earth
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: Market Prices

Post by Mordasius »

sk14 wrote: February 18th, 2024, 5:47 pm I'm glad that I wasn't crazy about the price issues with the skin.
I for one am more than glad that you reported the problem. I sold all my stocks and shares in November 2021 and don't use the MarketPrices or MyPortfolio skins anymore. If you hadn't reported the error with the prices then everyone who has downloaded the MarketPrices skin in the past few weeks would have been looking at gogo-gaga price values. I've already uploaded a corrected version of MarketPrices using the computed price to Deviant Art as I'm 99.9% sure that is going to show the correct prices all the time.

I'll update that version of MarketPrices to use Yingognito's RegExp after checking when the markets open on Tuesday as that does not involve adding any new measures to the skin.
User avatar
Mordasius
Posts: 1173
Joined: January 22nd, 2011, 4:23 pm
Location: GMT +8

Re: Market Prices

Post by Mordasius »

I waited for the markets to open, ran the debug skin and confirmed that the RegExp suggested by Yingognito gave Prices consistent with those being shown on CNBC.com. I then made the changes to MarketPrices.ini refreshed the skin and found that my IP had apparently been blocked.
I'm banned.png
I then went back to an older version of MarketPrices.ini that used individual page reads for each stock and low and behold no more blocking. ‘Oh what a tangled web we weave/When first we practice to deceive...'

Latest version on DA incorporates the RegExp suggested by Yingognito and should be good to go. Just take a breath or five between loading and refreshing the skin.
You do not have the required permissions to view the files attached to this post.
sk14
Posts: 25
Joined: February 9th, 2024, 7:21 pm

Re: Market Prices

Post by sk14 »

Thanks so much for updating the skin. Before I install it, just wondering if there are any lines I need to add to format all displays (price, change, percent change) to limit decimal places to 2.