Use graph(source, target) for undirected graph or digraph(source, target) for directed graph to define a graph. I assume you have an undirected graph since you mention a symmetrical adjacence matrix.
If the graph is defined properly in Matlab, you can calculate a lot of graph related stuff (adjacency, incidence, bfsearch, ...). You can also define the graph with edge-tables/node-tables see doc graph
% sources
source = [1 2 3 4 3 5];
% targets
target = [2 4 1 1 5 1];
% create graph (or digraph)
myGraph = graph(source, target);
% "full" adjacency matrix (default output is sparse)
full(myGraph.adjacency)
% plot the graph
plot(myGraph)
