Downloading Specific Files from Amazon S3 Bucket
8 次查看(过去 30 天)
显示 更早的评论
Hello,
I am trying to download some files from here: https://fcon_1000.projects.nitrc.org/indi/retro/MPI_LEMON.html. I am using Cyberduck to download files for 200+ people. The problem is that for each person I only need a specific file found within 2-3 subfolders. My current tactic is to just download everything and select the files I am interested in. The problem is that my internet connection is not that great and it will probably take me a week to do so.
Is there a way to do this matlab, so I can download only the specific files I want?
0 个评论
回答(1 个)
Samay Sagar
2024-8-30
To download specific files from a website using MATLAB, you can use the “websave” function. This function is generally used for downloading files when you have direct URLs.
If the files are nested within subfolders and you need to navigate through directories, you might need to perform web scraping to list and access the specific files.
If possible, find a pattern in the URLs of the files you want to download. This will allow you to construct the URLs programmatically.
Here is an approach you can follow to automate downloading specific files using MATLAB:
% Base URL
baseURL = "your_base_url";
% List of subjects and specific files
subjects = {'subject1', 'subject2', 'subject3'}; % Replace with actual subject IDs
filename = 'specific_file.ext'; % Replace with the actual filename you need
% Loop through each subject and download the specific file
for i = 1:length(subjects)
subjectURL = [baseURL, subjects{i}, '/path/to/subfolder/', filename];
outputFile = fullfile('local_directory', [subjects{i}, '_', filename]); % Save with a unique name
try
websave(outputFile, subjectURL);
fprintf('Downloaded: %s\n', outputFile);
catch
fprintf('Failed to download: %s\n', subjectURL);
end
end
Given your slow internet connection, ensure you have a stable connection when running the script. Consider implementing error handling to retry downloads in case of network issues.
For more information about “websave” you can refer the following documentation:
1 个评论
Orestis
2024-8-31
Hi, I ended up using a Python library to do it. If I need to do it again in the future I will check websave, thanks
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!