Bike Repositioning (Citi Bike) ============================== The Citi Bike scenario simulates the bike repositioning problem triggered by the one-way bike trips based on the public trip data from `Citi Bike `_. .. Citi Bike is New York City’s bike share system, which consists of a fleet of bikes that are locked into a network of docking stations throughout the city. The bikes can be unlocked from one station and returned to any other station in the system, making them ideal for one-way trips. People use bike share to commute to work or school, run errands, get to appointments or social engagements, and more. Since the demand for bikes and empty docks is dynamically changed during a day, and the bike flow between two stations are not equal in a same period, some stations suffer from severe bike shortages, while some have too much bikes and too few empty docks. In such a situation, the bike repositioning is essential to balance the bike's supply and demand. A good bike repositioning can not only meet the needs in the stations with heavy ride demand but also free the stations that do not have enough empty docks. Also, in the long run, a good bike repositioning can improve the bike useability, empower citizens' daily life, and reduce the carbon emission. Resource Flow ------------- In this scenario, the **bike** is the central resource. Two events will trigger the movement of the bike: * The first one is the trip requirement, which may cause the bike transfer from the source station to the destination station; * The second one is the repositioning operation. It is used to rebalance the bike distribution among stations. Bike Trip ^^^^^^^^^ In the citi bike scenario in MARO, the trip generation and the corresponding bike flow is defined as follows: * Given a fixed time interval, for each specific source-destination station pair, a trip requirement will arise according to a predefined distribution or the real trip data. It depends on the chosen topology. * If there are enough available bikes in the source station of the trip requirement, the required bike(s) will be unlocked and assigned to this trip. Otherwise, a shortage will be recorded in the source station. * The trip duration is read from the trip log if real trip data is used. Otherwise, the duration will be sampled from a specific random distribution. * At the end of the trip, the bike will be returned to the destination station. But if the destination does not have enough available docks, the bike will be returned to the nearest station with available docks. Bike Repositioning ^^^^^^^^^^^^^^^^^^ As for the repositioning operation, the simulator in MARO will regularly check the remaining bikes in each station and compare it with a predefined low watermark and high watermark. If the bike inventory is lower than the low watermark, the station will generate a ``Demand`` event to request the supply of bikes from its neighboring stations. Similarly, if the bike inventory is higher than the high watermark, the station will generate a ``Supply`` event to transport excess bikes to its neighboring stations. The low watermark and the high watermark is specified in the topology and can be customized based on different requirements. The target station candidates of the ``Supply`` and ``Demand`` events are selected by a predefined multi-layer filter in this scenario: #. The distance between the caller station and the neighboring stations will be used to filter and get a specific number of stations; #. The number of available bikes at each candidate station will be used to further filter on the candidate stations. For a ``Supply`` event, the stations with less bikes will be kept, while for a ``Demand`` event, the stations with more bikes will be kept; #. The future trip requirement of the target station will be the last filter. For a ``Supply`` event, the stations with more future trip requirement will be left in the final station candidate set, while the stations with less future trip requirement will be left for ``Demand`` event. The size of the candidate sets in each filter level is specified in the topology and can be customized based on different requirements. Once the target station candidate is filtered, the ``action scope`` for each candidate will also be calculated in the simulator and return to the decision agent together with some other information in the `DecisionEvent <#decisionevent-in-citi-bike>`_. For a ``Supply`` event, the bike inventory of the caller station and the number of available docks of the target station candidates will be attached. On the contrary, for a ``Demand`` event, the number of available docks of the source station and the bike inventory of the target station candidates will be attached. Based on the given target station candidates and the corresponding ``action scope``\ , the decision agent of the caller station should decide how many bikes to transfer to/request from the target station. We call a pair of ``(target station, bike number)`` a repositioning action. After an action taken, the destination station should wait for a certain period to get the bikes available for trip requirement. The action `lead time `_ is sampled from a predefined distribution. Topologies ---------- To provide an exploration road map from easy to difficult, two kinds of topologies are designed and provided in Citi Bike scenario. Toy topologies provide a super simplified environment for algorithm debugging, while the real topologies with real data from Citi Bike historical trips can present the real problem to users. Toy Topologies ^^^^^^^^^^^^^^ In toy topology, the generation of the trip requirements follows a stable pattern as introduced above. The detailed trip demand pattern are listed as below. And we hope that these toy topologies can provide you with some insights about this scenario. .. image:: ../images/scenario/citibike.toys.svg :target: ../images/scenario/citibike.toys.svg :alt: Citi Bike toy topologies **toy.3s_4t**\ : There are three stations in this topology. Every two minutes, there will be a trip requirement from S2 to S3 and a trip requirement from S3 to S2. At the same time, every two minutes, the system will generate trip requirement from S1 to S3 and from S1 to S2 with a fixed probability (80% and 20%, respectively). In this topology, the traffic flow between S2 and S3 is always equal, but station S1 is a super consumer with only bikes flow out. So the best repositioning policy in this topology is to reposition bikes from S2 and S3 to S1. It requires the active request action of S1 or the proactive transfer action of S2 and S3. **toy.4s_4t**\ : There are four stations in this topology. According to the global trip demand, there are more returned bikes than leaving bikes in station S1 and S3, while more leaving bikes than returned bikes in station S2 and S4. So the best repositioning policy in this topology is to reposition the excess bikes from S1 and S3 to S2 and S4. Furthermore, the cooperation between these stations is also necessary since only a proper allocation can lead to a globally optimal solution. **toy.5s_6t**\ : There are five stations in this topology. Although trip demand is more complex than the other two topologies above, we can still find that station S1 is a self-balancing station, station S2 and S5 have more returned bikes, and station S3 and S4 have more leaving bikes. Just like in topology toy.4s_4t, the best repositioning policy is to reposition excess bikes from S2 and S5 to S3 and S4 coordinately. Real Topologies ^^^^^^^^^^^^^^^ **ny.YYYYMM**\ : Different from the stable generation model in the toy topologies, the trip requirement in the topology ny.YYYYMM is generated based on the real trip data from `Citi Bike `_\ , which includes the source station, the destination station, and the duration of each trip. Besides, the total number of available bikes in this kind of topologies is counted from the real trip data of the specific month. Weighted by the the latest capacity of each stations, the available bikes are allocated to each station, which constitutes the initial bike inventory of each station. In this series of topologies, the definition of the bike flow and the trigger mechanism of repositioning actions are the same as those in the toy topologies. We provide this series of topologies to better simulate the actual Citi Bike scenario. Naive Baseline ^^^^^^^^^^^^^^ Below are the final environment metrics of the method *no repositioning* and *random repositioning* in different topologies. For each experiment, we setup the environment and test for a duration of 1 week. No Repositioning ~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - toy.3s_4t - 15,118 - 8,233 - 0 * - toy.4s_4t - 9,976 - 7,048 - 0 * - toy.5s_6t - 16,341 - 9,231 - 0 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.201801 - 48,089 - 2,688 - 0 * - ny.201802 - 126,374 - 8,814 - 0 * - ny.201803 - 138,952 - 10,942 - 0 * - ny.201804 - 161,443 - 10,349 - 0 * - ny.201805 - 323,375 - 29,081 - 0 * - ny.201806 - 305,971 - 26,412 - 0 * - ny.201807 - 254,715 - 19,669 - 0 * - ny.201808 - 302,589 - 26,352 - 0 * - ny.201809 - 313,002 - 28,472 - 0 * - ny.201810 - 339,268 - 24,109 - 0 * - ny.201811 - 263,227 - 21,485 - 0 * - ny.201812 - 209,102 - 15,876 - 0 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.201901 - 161,474 - 10,775 - 0 * - ny.201902 - 187,354 - 12,593 - 0 * - ny.201903 - 148,371 - 7,193 - 0 * - ny.201904 - 280,852 - 16,906 - 0 * - ny.201905 - 287,290 - 27,213 - 0 * - ny.201906 - 379,415 - 33,968 - 0 * - ny.201907 - 309,365 - 21,105 - 0 * - ny.201908 - 371,969 - 33,703 - 0 * - ny.201909 - 344,847 - 24,528 - 0 * - ny.201910 - 351,855 - 29,544 - 0 * - ny.201911 - 324,327 - 29,489 - 0 * - ny.201912 - 184,015 - 14,205 - 0 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.202001 - 169,304 - 12,449 - 0 * - ny.202002 - 206,105 - 14,794 - 0 * - ny.202003 - 235,986 - 15,436 - 0 * - ny.202004 - 91,810 - 2,348 - 0 * - ny.202005 - 169,412 - 5,231 - 0 * - ny.202006 - 197,883 - 7,608 - 0 Random Repositioning ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - toy.3s_4t - 15,154 - 8,422 :math:`\pm` 11 - 449 :math:`\pm` 22 * - toy.4s_4t - 10,186 - 4,371 :math:`\pm` 72 - 3,392 :math:`\pm` 83 * - toy.5s_6t - 16,171 - 7,513 :math:`\pm` 40 - 3,242 :math:`\pm` 71 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.201801 - 48,089 - 6,693 :math:`\pm` 138 - 445,996 :math:`\pm` 6,756 * - ny.201802 - 126,374 - 21,418 :math:`\pm` 120 - 446,564 :math:`\pm` 3,505 * - ny.201803 - 138,952 - 22,121 :math:`\pm` 272 - 448,259 :math:`\pm` 1,831 * - ny.201804 - 161,443 - 22,201 :math:`\pm` 194 - 453,705 :math:`\pm` 3,697 * - ny.201805 - 323,375 - 54,365 :math:`\pm` 538 - 470,771 :math:`\pm` 5,337 * - ny.201806 - 305,971 - 49,876 :math:`\pm` 1,091 - 481,443 :math:`\pm` 6,981 * - ny.201807 - 254,715 - 46,199 :math:`\pm` 204 - 483,788 :math:`\pm` 982 * - ny.201808 - 302,589 - 53,679 :math:`\pm` 433 - 485,137 :math:`\pm` 2,557 * - ny.201809 - 313,002 - 61,432 :math:`\pm` 75 - 474,851 :math:`\pm` 2,908 * - ny.201810 - 339,268 - 64,269 :math:`\pm` 600 - 461,928 :math:`\pm` 1,018 * - ny.201811 - 263,227 - 40,440 :math:`\pm` 239 - 467,050 :math:`\pm` 6,595 * - ny.201812 - 209,102 - 26,067 :math:`\pm` 234 - 457,173 :math:`\pm` 6,444 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.201901 - 161,474 - 19,295 :math:`\pm` 155 - 444,445 :math:`\pm` 2,287 * - ny.201902 - 187,354 - 23,875 :math:`\pm` 282 - 456,888 :math:`\pm` 362 * - ny.201903 - 148,371 - 12,451 :math:`\pm` 312 - 409,226 :math:`\pm` 5,392 * - ny.201904 - 280,852 - 29,591 :math:`\pm` 170 - 464,671 :math:`\pm` 6,148 * - ny.201905 - 287,290 - 44,199 :math:`\pm` 542 - 485,077 :math:`\pm` 6,140 * - ny.201906 - 379,415 - 51,396 :math:`\pm` 256 - 503,503 :math:`\pm` 4,742 * - ny.201907 - 309,365 - 33,861 :math:`\pm` 643 - 500,443 :math:`\pm` 4,314 * - ny.201908 - 371,969 - 51,319 :math:`\pm` 417 - 516,684 :math:`\pm` 1,400 * - ny.201909 - 344,847 - 34,532 :math:`\pm` 466 - 476,965 :math:`\pm` 3,932 * - ny.201910 - 351,855 - 37,828 :math:`\pm` 502 - 496,135 :math:`\pm` 4,167 * - ny.201911 - 324,327 - 34,745 :math:`\pm` 427 - 484,599 :math:`\pm` 8,771 * - ny.201912 - 184,015 - 20,119 :math:`\pm` 110 - 437,311 :math:`\pm` 5,936 | .. list-table:: :header-rows: 1 * - Topology - Total Requirement - Resource Shortage - Repositioning Number * - ny.202001 - 169,304 - 17,152 :math:`\pm` 218 - 476,821 :math:`\pm` 1,052 * - ny.202002 - 206,105 - 24,223 :math:`\pm` 209 - 480,012 :math:`\pm` 1,547 * - ny.202003 - 235,986 - 23,749 :math:`\pm` 654 - 458,536 :math:`\pm` 1,457 * - ny.202004 - 91,810 - 3,349 :math:`\pm` 48 - 326,817 :math:`\pm` 3.131 * - ny.202005 - 169,412 - 10,177 :math:`\pm` 216 - 378,038 :math:`\pm` 2,429 * - ny.202006 - 197,883 - 11,741 :math:`\pm` 170 - 349,932 :math:`\pm` 4,375 Quick Start ----------- Data Preparation ^^^^^^^^^^^^^^^^ To start the simulation of Citi Bike scenario, users have two options for the data preparation: * If the topology data is not generated in advance, the system will automatically download and process the relevant data when the environment is created. The data will be stored in a temporary folder and be automatically deleted after the experiment. * Before creating the environment, users can also manually download and generate relevant data. This approach will save you a lot of time if you need to conduct several experiments on the same topology. Therefore, we encourage you to generate the relevant data manually first. The following is the introduction to related commands: Environment List Command ~~~~~~~~~~~~~~~~~~~~~~~~ The data environment ``list`` command is used to list the environments that need the data files generated before the simulation. .. code-block:: sh maro env data list scenario: citi_bike, topology: ny.201801 scenario: citi_bike, topology: ny.201802 scenario: citi_bike, topology: ny.201803 scenario: citi_bike, topology: ny.201804 scenario: citi_bike, topology: ny.201805 scenario: citi_bike, topology: ny.201806 ... Generate Command ~~~~~~~~~~~~~~~~ The data ``generate`` command is used to automatically download and build the specified predefined scenario and topology data files for the simulation. Currently, there are three arguments for the data ``generate`` command: * ``-s``\ : required, used to specify the predefined scenario. Valid scenarios are listed in the result of `environment list command <#environment-list-command>`_. * ``-t``\ : required, used to specify the predefined topology. Valid topologies are listed in the result of `environment list command <#environment-list-command>`_. * ``-f``\ : optional, if set, to force to re-download and re-generate the data files and overwrite the already existing ones. .. code-block:: sh maro env data generate -s citi_bike -t ny.201802 The data files for citi_bike-ny201802 will then be downloaded and deployed to ~/.maro/data/citibike/_build/ny201802. For the example above, the directory structure should be like: .. code-block:: sh |-- ~/.maro |-- data | |-- citi_bike | |-- .build # bin data file | |-- [topology] # topology | |-- .source | |-- .download # original data file | |-- .clean # cleaned data file |-- temp # download temp file Build Command ~~~~~~~~~~~~~ The data ``build`` command is used to build the CSV data files to binary data files that the simulator needs. Currently, there are three arguments for the data ``build`` command: * ``--meta``\ : required, used to specify the path of the meta file. The source columns that to be converted and the data type of each columns should be specified in the meta file. * ``--file``\ : required, used to specify the path of the source CSV data file(s). If multiple source CSV data files are needed, you can list all the full paths of the source files in a specific file and use a ``@`` symbol to specify it. * ``--output``\ : required, used to specify the path of the target binary file. .. code-block:: sh maro data build --meta ~/.maro/data/citibike/meta/trips.yml --file ~/.maro/data/citibike/source/_clean/ny201801/trip.csv --output ~/.maro/data/citibike/_build/ny201801/trip.bin Environment Interface ^^^^^^^^^^^^^^^^^^^^^ Before starting interaction with the environment, we need to know the definition of ``DecisionEvent`` and ``Action`` in Citi Bike scenario first. Besides, you can query the environment `snapshot list <../key_components/data_model.html#advanced-features>`_ to get more detailed information for the decision making. DecisionEvent ~~~~~~~~~~~~~ Once the environment need the agent's response to reposition bikes, it will throw an ``DecisionEvent``. In the scenario of Citi Bike, the information of each ``DecisionEvent`` is listed as below: * **station_idx** (int): The id of the station/agent that needs to respond to the environment. * **tick** (int): The corresponding tick. * **frame_index** (int): The corresponding frame index, that is the index of the corresponding snapshot in the environment snapshot list. * **type** (DecisionType): The decision type of this decision event. In Citi Bike scenario, there are 2 types: * ``Supply`` indicates there is too many bikes in the corresponding station, so it is better to reposition some of them to other stations. * ``Demand`` indicates there is no enough bikes in the corresponding station, so it is better to reposition bikes from other stations. * **action_scope** (dict): A dictionary that maintains the information for calculating the valid action scope: * The key of these item indicate the station/agent ids. * The meaning of the value differs for different decision type: * If the decision type is ``Supply``\ , the value of the station itself means its bike inventory at that moment, while the value of other target stations means the number of their empty docks. * If the decision type is ``Demand``\ , the value of the station itself means the number of its empty docks, while the value of other target stations means their bike inventory. Action ~~~~~~ Once we get a ``DecisionEvent`` from the environment, we should respond with an ``Action``. Valid ``Action`` could be: * ``None``\ , which means do nothing. * A valid ``Action`` instance, including: * **from_station_idx** (int): The id of the source station of the bike transportation. * **to_station_idx** (int): The id of the destination station of the bike transportation. * **number** (int): The quantity of the bike transportation. Example ^^^^^^^ Here we will show you a simple example of interaction with the environment in random mode, we hope this could help you learn how to use the environment interfaces: .. code-block:: python from maro.simulator import Env from maro.simulator.scenarios.citi_bike.common import Action, DecisionEvent, DecisionType import random # Initialize an environment of Citi Bike scenario, with a specific topology. # In CitiBike, 1 tick means 1 minute, durations=1440 here indicates a length of 1 day. # In CitiBike, one snapshot will be maintained every snapshot_resolution ticks, # snapshot_resolution=30 here indicates 1 snapshot per 30 minutes. env = Env(scenario="citi_bike", topology="toy.3s_4t", start_tick=0, durations=1440, snapshot_resolution=30) # Query for the environment summary, the business instances and intra-instance attributes # will be listed in the output for your reference. print(env.summary) metrics: object = None decision_event: DecisionEvent = None is_done: bool = False action: Action = None num_episode = 2 for ep in range(num_episode): # Gym-like step function. metrics, decision_event, is_done = env.step(None) while not is_done: past_2hour_frames = [ x for x in range(decision_event.frame_index - 4, decision_event.frame_index) ] decision_station_idx = decision_event.station_idx intr_station_infos = ["trip_requirement", "bikes", "shortage"] # Query the snapshot list of this environment to get the information of # the trip requirements, bikes, shortage of the decision station in the past 2 hours. past_2hour_info = env.snapshot_list["stations"][ past_2hour_frames : decision_station_idx : intr_station_infos ] if decision_event.type == DecisionType.Supply: # Supply: the value of the station itself means the bike inventory. self_bike_inventory = decision_event.action_scope[decision_event.station_idx] # Supply: the value of other stations means the quantity of empty docks. target_idx_dock_tuple_list = [ (k, v) for k, v in decision_event.action_scope.items() if k != decision_event.station_idx ] # Randomly choose a target station weighted by the quantity of empty docks. target_idx, target_dock = random.choices( target_idx_dock_tuple_list, weights=[item[1] for item in target_idx_dock_tuple_list], k=1 )[0] # Generate the corresponding random Action. action = Action( from_station_idx=decision_event.station_idx, to_station_idx=target_idx, number=random.randint(0, min(self_bike_inventory, target_dock)) ) elif decision_event.type == DecisionType.Demand: # Demand: the value of the station itself means the quantity of empty docks. self_available_dock = decision_event.action_scope[decision_event.station_idx] # Demand: the value of other stations means their bike inventory. target_idx_inventory_tuple_list = [ (k, v) for k, v in decision_event.action_scope.items() if k != decision_event.station_idx ] # Randomly choose a target station weighted by the bike inventory. target_idx, target_inventory = random.choices( target_idx_inventory_tuple_list, weights=[item[1] for item in target_idx_inventory_tuple_list], k=1 )[0] # Generate the corresponding random Action. action = Action( from_station_idx=target_idx, to_station_idx=decision_event.station_idx, number=random.randint(0, min(self_available_dock, target_inventory)) ) else: action = None # Drive the environment with the random action. metrics, decision_event, is_done = env.step(action) # Query for the environment business metrics at the end of each episode, # it is usually users' optimized object (usually includes multi-target). print(f"ep: {ep}, environment metrics: {env.metrics}") env.reset() Jump to `this notebook `_ for a quick experience.