Function precedence in multiple nested Matlab projects

4 次查看（过去 30 天）
Jan Kappen
Jan Kappen 2024-2-7
评论： Jan Kappen 2024-4-2
Matlab projects can add other projects as references. I started to use this to modularize my Matlab libraries. In fact, every git repo belongs to one Matlab package/namespace and gets consumed by an application. I started to use buildinspace/peru: a generic package manager, for including other people's code in your projects (github.com) for source code based dependency management (no, I do not want to use submodules).
That means, it ends up with a folder structure like
myApp
- src
-- myApp.m % needs my.namespace.lib1.m and my.othernamespace.lib2
- myAppPrj.prj % references lib-ml-lib1.prj and lib-ml-lib2.prj, does not set anything manually to the path
- resources % ML project resources
- lib
-- lib-ml-lib1
--- lib-ml-lib1.prj % adds src folder of lib1 to path
--- resources % ML project resources lib1
--- src/
---- +my/+othernamespace/lib1.m
-- lib-ml-lib2
--- lib-ml-lib2.prj % adds src folder of lib2 to path
--- resources % ML project resources lib2
--- src/
---- +my/+namespace/lib2.m
I hope the structure is clear; myApp.m depends on lib1.m and lib2.m which are functions or classes in nested packages.
This worked fine until lib1 and lib2 started to depend on lib3, but in potentially different versions. peru successfully copies the requested versions of lib3 into the lib folders in lib1 and lib2, but it seems to be random which version is chosen and I don't understand what's going on.
New situation:
myApp
- src
-- myApp.m % needs my.namespace.lib1.m and my.othernamespace.lib2
- myAppPrj.prj % references lib-ml-lib1.prj and lib-ml-lib2.prj, does not set anything manually to the path
- resources % ML project resources
- lib
-- lib-ml-lib1
--- lib-ml-lib1.prj % adds src folder of lib1 to path
--- resources % ML project resources lib1
--- src/
---- +my/+othernamespace/lib1.m
--- lib
---- lib-ml-lib3 % lib 3 in version 1.x
----- lib-ml-lib3.prj % adds src folder of lib3 to path
----- src/
------ +my/+otherothernamespace/lib3.m
-- lib-ml-lib2
--- lib-ml-lib2.prj % adds src folder of lib2 to path
--- resources % ML project resources lib2
--- src/
---- +my/+namespace/lib2.m
--- lib
---- lib-ml-lib3 % lib 3 in version 2.x
----- lib-ml-lib3.prj % adds src folder of lib3 to path
----- src/
------ +my/+otherothernamespace/lib3.m
Some questions
  • Is the first found function in the pathtool used or the last one?
  • how do referenced projects work, does every project have its own path or does everything go into a global one?
  • how to explain that which("my.otherothernamespace.lib3", "-all") from the myApp (which does not directly depend on lib3) level finds the lib in the lib-ml-lib2 folder, i.e. in version 2.x but the profiler shows that lib-ml-lib2 uses the lib3 in lib-ml-lib1 folder (!), i.e. in version 1.x although its local lib folder contains the lib3 in version 2.x?
  • What whould you do to solve that? It's not possible to avoid nested libs if I want to do proper dependency control
Thanks!
Jan

请先登录，再回答此问题。

回答（1 个）

Rishi
Rishi 2024-2-28
Hi Jan,
I understand from your question that you want explanations regarding function precedence in MATLAB search path. Below are the answer to your questions:
1. The first file or function found in the pathtool is used. To verify this, you can use the 'which' function. In the MATLAB Command Winodw, type the following:
which filename
This returns the file which will be used. You can learn more about it from the given documentation:
https://www.mathworks.com/help/matlab/ref/which.html#:~:text=If%20item%20is%20an%20overloaded%20function%20or%20method%2C%20then%20which%20item%20returns%20only%20the%20path%20of%20the%20first%20function%20or%20method%20found.
2. Referenced projects in MATLAB add their paths to the global MATLAB path. This path is shared across the entire MATLAB environment, which means that all projects contribute to and use the same global path.
3. The 'which' function, when used with '-all' flag, prints out all of the paths of all item with the requested name. Hence, the below line will show all instances of the function 'my.otherothernamespace.lib3' currently on path:
which("my.otherothernamespace.lib3", "-all")
More about it can be read from the below documentation:
https://www.mathworks.com/help/matlab/ref/which.html#:~:text=___-,%2Dall,-displays%20the%20paths
The Profiler shows that 'lib-ml-lib2' uses the 'lib3' in 'lib-ml-lib1' folder because this version of the file is higher up on the MATLAB path than the 'lib-ml-lib2' folder.
4. Here are a couple of possible solutions to this:
Version Control within Namespaces: Ensure that each version of lib3 is placed in a different namespace corresponding to its version. This way, lib1 and lib2 can explicitly reference the correct version of lib3.
Project Initialization Scripts: Write custom initialization scripts for each project that set up the path correctly by removing any conflicting paths and then adding the required version of lib3. This ensures that the correct version is used during the project's execution. You can do this using the functions 'addpath' and 'rmpath'. You can learn more about these functions from the below documentations:
https://www.mathworks.com/help/matlab/ref/addpath.html
https://www.mathworks.com/help/matlab/ref/rmpath.html
Hope this helps!
  3 个评论
显示 1更早的评论
Rishi
Rishi 2024-2-28
编辑：Rishi 2024-2-28
You can create a wrapper function in which you can pass in the namespace as an argument, and then use the 'eval' function to evaluate the function. You can learn more about the 'eval' function from the below documentation:
https://www.mathworks.com/help/matlab/ref/eval.html
I have implemented the code below:
function output = libraryWrapper(version,inputArg1,inputArg2)
evalString = ['my.', version, '.lib3(', num2str(inputArg1),',', num2str(inputArg2), ');'];
output = eval(evalString);
end
My folders are as follows:
Main
- Sub1
-- +my
--- +ns
---- lib3.m
- Sub2
-- +my
--- +ns1
---- lib3.m
In this sample implementation, the 'lib3' function in 'ns' adds two numbers, whereas 'lib3' in 'ns1' subtracts them.
I have attached the output for both of the cases.
While publishing your libraries as MATLAB toolbox will directly solve your issue as toolboxes are added to the global path, but it can be helpful in packaging, versioning and ease of distribution.
Jan Kappen
Jan Kappen 2024-4-2
Dear @Rishi,
thanks for your answer but I'm afraid that's not the right approach. eval is definetely not recommended and I can't wrap everything into different namespaces. That won't work :(
An ideal solution would be that every ML project only sees files that belong to its path, not to the parent's project path.
Does anyone know if MathWorks is working on something like that? A global path breaks everything here.
Or am I tackling the problem from a wrong side? I just do not want to update all ML projects that depent on a certain library when that one gets updated.
Simple example:
  • lib_io depends on lib_system v1.1
  • my_app depends on lib_io and lib_system v1.1
All dependency management happens on code level using peru (similar, but imo better than git submodules), i.e. every lib_io and my_app both contain a lib folder with the correctly checked out version v1.1. Now I make an update to lib_system and tag it v1.2, because my_app needs a new functionality.
New situation:
  • lib_io depends on lib_system v1.1
  • my_app depends on lib_io and lib_system v1.2
Now, it's not clear if v1 or v1.2 is used, because both locations are in the global path when the corresponding projects are loaded.
Of course I could update lib_io to use lib_system v1.2, too. But that requires to check out the lib_io's code, update the manifest file where I declare the dependency to lib_systems to use lib_systems v1.2, check that change in, increase version of lib_io and ultimately update my_app to use the latest lib_io.
I need to do that for loooots of libraries - in lockstep. Just to have a new functionality in my_app. For sure this could be scripted, but it just works as long as there are minor and backward compatible changes. What if there are incompatible changes?
Ultimately, I think everything points to the direction that MATLAB needs a proper built-in dependency manager like pip, but right now there's none coming and I wonder how to fix it on my own.
Any ideas?
Thanks,
Jan

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Source Control in Projects 的更多信息

标签

产品


版本

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by