Hi Eric,
I understand you're attempting to access the MATLAB forums page using the ‘webread’ function with specific query parameters. However, directly utilizing the URL for the desired page with ‘webread’ results in the following error.
'''
The connection to URL 'https://www.mathworks.com/matlabcentral/answers/?product_base_code%5B%5D=Simulink&sort=views+desc' timed out after 5.000 seconds. The reason is "Waiting for response header". Perhaps the server is not responding or weboptions.Timeout needs to be set to a higher value.
'''
Increasing the ‘Timeout’ value using ‘weboptions’ as suggested in the error works and fetches the correct page. Here is the code for same.
webopts = weboptions('Timeout', 10);
data = webread('https://in.mathworks.com/matlabcentral/answers/?page=1&sort=views+desc&term=product_base_code%3ASL', webopts);
To incorporate a page index into the URL within a for loop, you can utilize the ‘strcat’ function.
for index = 1:5
webopts = weboptions('Timeout', 10);
url = strcat('https://in.mathworks.com/matlabcentral/answers/?page=',num2str(index),'&sort=views+desc&term=product_base_code%3ASL');
data = webread(url, webopts);
end
The above code will fetch 5 pages from MATLAB central. You can use ‘websave’ if you wish to save the fetched pages in HTML files.
Please go through following MATLAB documentation to learn more about ‘weboptions’, ‘strcat’ and ‘websave’:
- https://www.mathworks.com/help/matlab/ref/weboptions.html
- https://www.mathworks.com/help/matlab/ref/strcat.html
- https://www.mathworks.com/help/matlab/ref/websave.html
Hope This helps!
Thank You
Kanishk Singhal
