How to trigger an excel file download in chrome from Matlab. (403 with message "Forbidden" )

Question

Ernesto 2024-12-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2172285-how-to-trigger-an-excel-file-download-in-chrome-from-matlab-403-with-message-forbidden

评论： Walter Roberson 2024-12-22

Hello Dear Matlabers!

I coded a pretty simple code for downloading excel-files from the US Securities and Exchange Commission online site EDGAR (sec.gov) some time ago and it worked just fine. It went over thousends of files waiting 10 seconds each time as requested from the web site. I used the function

websave(filename,url);

The code:

url=append('https://www.sec.gov/Archives/edgar/data/',string(Q_10(b).Cat.EDGAR(i,1).cik),'/',string(erase(M(x),'-')),'/Financial_Report.xlsx');
                try
                    filename = append('',string(Q_10(b).Cat.IDName(i)),'_',string(erase(M(x),'-')),'.xlsx');
                    websave(filename,url);
                    x=x+1;
                    pause(1/10); %Current guidelines limit each user to a total of no more than 10 requests per second%
                catch

After some months I realized I had to many files and wanted to download only some of the files. (don`t aks why i need them to download again). Now the function does not seam to work any more. I get:

Error using matlab.internal.webservices.HTTPConnector/copyContentToFile (line 389)
The server returned the status 403 with message "Forbidden" in response to the request to URL
https://www.sec.gov/Archives/edgar/data/831259/000083125923000037/Financial_Report.xlsx.
Error in websave (line 107)
copyContentToFile(connection, filename); 

The funny thing is that when I use the url on my Chrome browser:

url = "https://www.sec.gov/Archives/edgar/data/831259/000083125923000037/Financial_Report.xlsx" .

It works just fine and the files gets downloaded. I have tried to fix the problem for months now. And I have lost hope.... =(

I even tried to fix the problem with ChatGPT and downloaded selenium and java toolbox to try to configure the ChromeOptions from matlab but I can't get it to run. I am using a MacBook Pro with Apple M1 chip. (please spare me the jokes on mac) and a MATLAB R2021b home use licenze. I have not renewed the "Software Maintenance" and can't seem to add the toolboxes (selenium-java-4.27.0 & chromedriver-mac-arm64) in the Add-On Explorer. Think my promblem is much easier to fix than adding toolboxes specially if I am able to trigger the automatic download when typing the url directly on chrome. thank you for your help!! best regards Ernesto

not that I want to use this toolboxes but this is my code for that:

cd '/Users/....../MathWorks/MATLAB Add-Ons/Toolboxes/selenium-java-4.27.0'
% Import the required packages
import org.openqa.selenium.chrome.ChromeDriver.*
import org.openqa.selenium.chrome.ChromeOptions.*
import org.openqa.selenium.remote.DesiredCapabilities.*
% Specify the path to ChromeDriver
chromeDriverPath = '/Users/mondragon/Library/Application Support/MathWorks/MATLAB Add-Ons/Toolboxes/chromedriver-mac-arm64'
javaaddpath('/Users/mondragon/Library/Application Support/MathWorks/MATLAB Add-Ons/Toolboxes/selenium-java-4.27.0');
java.lang.System.setProperty('webdriver.chrome.driver', chromeDriverPath);
cd '/Users/mondragon/Library/Application Support/MathWorks/MATLAB Add-Ons/Toolboxes/chromedriver-mac-arm64'
% Create ChromeOptions to configure Chrome
ChromeOptions()
prefs = java.util.HashMap();
prefs.put('download.default_directory', 'C:\\your\\desired\\path'); % Set your desired download directory here
chromeOptions.setExperimentalOption('prefs', prefs);
% Initialize ChromeDriver with options
driver = ChromeDriver(chromeOptions);

result:

Unrecognized function or variable 'ChromeOptions'.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Walter Roberson 2024-12-22

Possibly the remote side needs to see a UserAgent header.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Divyajyoti Nayak 2024-12-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2172285-how-to-trigger-an-excel-file-download-in-chrome-from-matlab-403-with-message-forbidden#answer_1556238

编辑：Divyajyoti Nayak 2024-12-22

Hi @Ernesto Lopez Mondragon,

The status code given in the error is 403 which means that the request was recognized by the server but it was denied due to lack of authorization. This change is most likely done by the people handling the website. Since you are able to dowload directly from the browser, a workaround could be to open the URL in the browser via the 'web' function and then rename the files accordingly, with the 'movefile' function

'web' function: https://www.mathworks.com/help/matlab/ref/web.html

'movefile' function: https://www.mathworks.com/help/matlab/ref/movefile.html?s_tid=doc_ta