Taxi datset - lat lon colocation

Question

LeoAiE 2022-11-11

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1848973-taxi-datset-lat-lon-colocation

编辑： LeoAiE 2022-11-12

taxi_data.zip

Hi everyone,

I’m practicing data science with NY Taxi data (https://www.kaggle.com/competitions/nyc-taxi-trip-duration/data). I want to check if certain taxis were collocated within a short distance from each other.

The task I’m trying to accomplish is:

Measure the distance for each taxi (against all other taxis)
If the taxis were within 20 m of each other – just an example
Check the date column associate with those lat lon matches
If the date match, meaning the two taxis visited the general area on the same date
Return the two unique IDs of those taxis, the date, the distance, and the lat lon of both of them

Here is my progress and I would really appreciate your help.

taxi = readtable('yellow_tripdata_2015-01.csv') % reading the data 
taxi.uniqueID = randi([1000000,9999999],[height(taxi),1]) % Creating random hopefully unique ID :)
% measuring the distance from pick up to drop off
taxi.distance_travelled_in_M = deg2km(distance(taxi.pickup_latitude, taxi.pickup_longitude, taxi.dropoff_latitude, taxi.dropoff_longitude))
% spliting date and time in two diffrent columns 
taxi.date = datestr(datetime(taxi.tpep_pickup_datetime),'dd/mm/yyyy') 
taxi.time = datestr(datetime(taxi.tpep_pickup_datetime),'HH:MM:SS')
% In this loop I took each lat lon point and ran it agaist the rest of the
% data- probably not an efficient way - open to suggestions 
% then converting the distance to meters 
% checking if the distance within for example 20 meters 
% and the date is the same - this step is not implemented yet , need help
% return the unique ids of both taxis, date of that event, the location
% (lat,lon), the distacne how far from each other 
for idx = 1:length(taxi.pickup_latitude)
    my_dist = distance(taxi.pickup_latitude(idx), taxi.pickup_longitude(idx), taxi.pickup_latitude, taxi.pickup_longitude); % distqances in degrees 
    my_dist = deg2km(my_dist); % distance in meters 
    if my_dist < 2
        pos_results = [pos_results;my_dist;taxi(idx:idx, ["uniqueID", "date", "pickup_latitude", "pickup_longitude"])]
    end 
end

2 个评论
显示无隐藏无

Walter Roberson 2022-11-12

在 MATLAB Online 中打开

    my_dist = distance(taxi.pickup_latitude(idx), taxi.pickup_longitude(idx), taxi.pickup_latitude, taxi.pickup_longitude); % distqances in degrees 

Not really degrees. You are doing Euclidean distance calculations on a non-linear surface. The result is only meaningful if the data is recorded near the equator. New York City is 40.7128N and cosd() of that is about 0.76 so degrees longitude there are only roughly 3/4 of the distance of a degree latitude.

Have you considered calculating Great Circle Distance ?

LeoAiE 2022-11-12

Hi @Walter Roberson

Thank you for your comments! I have not. I will research how to calculate great circle disctance!

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2022-11-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1848973-taxi-datset-lat-lon-colocation#answer_1097648

编辑：Walter Roberson 2022-11-12

Instead, do a rangesearch which is a knnsearch by distance.

Either do a custom distance calculation of Great Circle Distance, or else divide the longitudes by cosd() of the latitudes in order to adjust the two of them to be on the same scale, and then use Euclidean.

3 个评论
显示 1更早的评论隐藏 1更早的评论

LeoAiE 2022-11-12

在 MATLAB Online 中打开

@Walter Roberson still trying to process your method in my head lol very high level, but I found a function that possibly takes care of the distance. I'm still not sure how to implement the rest of my steps

Steve Ratts (2022). Compute the Great Circle Distance between two points (https://www.mathworks.com/matlabcentral/fileexchange/23026-compute-the-great-circle-distance-between-two-points), MATLAB Central File Exchange. Retrieved November 12, 2022.

function d = greatCircleDistance(phi_s, lambda_s, phi_f, lambda_f, r)
% compute the great circle distance given lat and long for two points
% optionally, a fifth parameter (r) can be specified. If this paramter
% isn't specified it's assumed to be the mean radius of the earth. The
% calculation is done using the Vincenty formula.
%
% INPUTS:
% phi_s    = latitude of the standpoint (base) [rad]
% lambda_s = longitude of the standpoint (base) [rad]
% phi_f    = latitude of the forepoint (destination) [rad]
% lambda_f = longitude of the forepoint (destination) [rad]
% r        = radius of the sphere [units determine units of d]
%
% OUTPUT:
% d        = great circle distance from standpoint to forepoint
%
% See http://en.wikipedia.org/wiki/Great-circle_distance
% If no arguments, bail out
if nargin < 4
    fprintf('Usage: greatCircleDistance(phi_s, lambda_s, phi_f, lambda_f, r)\n')
    return
end
% If no radius supplied, assume the mean radius of the earth in km
if nargin < 5
    r = 6371.01; % km
end
% convert from degrees minutes seconds to radians as needed
if isstruct(phi_s) || (length(phi_s) > 1 && ~isstruct(phi_s))
    phi_s = dms2r(phi_s);
end
if isstruct(lambda_s) || (length(lambda_s) > 1 && ~isstruct(lambda_s))
    lambda_s = dms2r(lambda_s);
end
if isstruct(phi_f) || (length(phi_f) > 1 && ~isstruct(phi_f))
    phi_f = dms2r(phi_f);
end
if isstruct(lambda_f) || (length(lambda_f) > 1 && ~isstruct(lambda_f))
    lambda_f = dms2r(lambda_f);
end
% Compute Delta lambda (delta longitude)
Delta_lambda = lambda_f - lambda_s;
% Compute Delta sigma (central angle)
Delta_sigma = atan2(sqrt((cos(phi_f)*sin(Delta_lambda))^2 + (cos(phi_s)*sin(phi_f) - sin(phi_s)*cos(phi_f)*cos(Delta_lambda))^2), ...
    sin(phi_s)*sin(phi_f) + cos(phi_s)*cos(phi_f)*cos(Delta_lambda));
d = r*Delta_sigma;
function r = dms2r(dms)
if isstruct(dms)
    r = sign(dms.deg)*(abs(dms.deg) + (dms.min + dms.sec/60)/60)*pi/180;
elseif length(dms) == 3
    r = sign(dms(1))*(abs(dms(1)) + (dms(2) + dms(3)/60)/60)*pi/180;
elseif length(dms) == 2
    r = sign(dms(1))*(abs(dms(1)) + dms(2)/60)*pi/180;
else
    r = nan;
end

Walter Roberson 2022-11-12

https://www.mathworks.com/help/map/ref/distance.html if you have the mapping toolbox

LeoAiE 2022-11-12

编辑：LeoAiE 2022-11-12

在 MATLAB Online 中打开

Yes I do have it and thats why I use the distance function in my original post but I guess I have to specify somehting like this

s = referenceSphere('Earth')
distance(taxi.pickup_latitude(1), taxi.pickup_longitude(1), taxi.pickup_latitude(2), taxi.pickup_longitude(2),s)

The issue now how to check the date column associate with those lat lon matches and If the date match, meaning the two taxis visited the general area on the same date then return the two unique IDs of those taxis, the date, the distance, and the lat lon of both of them

请先登录，再进行评论。