FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881233         	|  731.88         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  34.66          	|  20             	|  693.2          	|  94.715         	|
|  run_training_batch                                                                                                                                                                                         	|  0.027262       	|  20000          	|  545.23         	|  74.497         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.027018       	|  20000          	|  540.37         	|  73.833         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.0096714      	|  20000          	|  193.43         	|  26.429         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0037108      	|  20000          	|  74.216         	|  10.14          	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.054          	|  48             	|  50.593         	|  6.9128         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0018232      	|  20000          	|  36.464         	|  4.9823         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  25.925         	|  1              	|  25.925         	|  3.5423         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00044184     	|  20000          	|  8.8369         	|  1.2074         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.16265        	|  20             	|  3.2529         	|  0.44446        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00011227     	|  20048          	|  2.2507         	|  0.30753        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.0785e-05     	|  20048          	|  1.4191         	|  0.1939         	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.012168       	|  48             	|  0.58407        	|  0.079803       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.7628e-05     	|  20000          	|  0.35255        	|  0.048171       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.4092e-05     	|  20000          	|  0.28184        	|  0.038509       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.24e-05       	|  20000          	|  0.248          	|  0.033886       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.2823e-06     	|  20000          	|  0.085647       	|  0.011702       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.037441       	|  2              	|  0.074882       	|  0.010231       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.3365e-06     	|  20000          	|  0.06673        	|  0.0091176      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.1451e-06     	|  20000          	|  0.062902       	|  0.0085945      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.0334e-06     	|  20000          	|  0.060668       	|  0.0082894      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.6826e-06     	|  20000          	|  0.053651       	|  0.0073306      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.6366e-06     	|  20000          	|  0.052731       	|  0.0072049      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.5398e-06     	|  20000          	|  0.050796       	|  0.0069404      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.4406e-06     	|  20000          	|  0.048811       	|  0.0066693      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3399e-06     	|  20000          	|  0.046797       	|  0.0063941      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.317e-06      	|  20000          	|  0.046339       	|  0.0063315      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2571e-06     	|  20000          	|  0.045143       	|  0.006168       	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.2422e-06     	|  20000          	|  0.044845       	|  0.0061273      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.0902e-06     	|  20000          	|  0.041803       	|  0.0057118      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.0787e-06     	|  20000          	|  0.041575       	|  0.0056805      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.0314e-06     	|  20000          	|  0.040627       	|  0.005551       	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.0266e-06     	|  20000          	|  0.040532       	|  0.0055381      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.017e-06      	|  20000          	|  0.040341       	|  0.0055119      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  1.9893e-06     	|  20000          	|  0.039786       	|  0.0054361      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  1.9766e-06     	|  20000          	|  0.039532       	|  0.0054015      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  1.954e-06      	|  20000          	|  0.039081       	|  0.0053398      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.929e-06      	|  20000          	|  0.03858        	|  0.0052713      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9178e-06     	|  20000          	|  0.038356       	|  0.0052407      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.1037e-06     	|  20048          	|  0.022126       	|  0.0030232      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00042339     	|  48             	|  0.020323       	|  0.0027768      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.0103e-06     	|  20000          	|  0.020206       	|  0.0027608      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  9.7855e-07     	|  20048          	|  0.019618       	|  0.0026805      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  9.5576e-07     	|  20000          	|  0.019115       	|  0.0026118      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.271e-07      	|  20000          	|  0.018542       	|  0.0025335      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.8992e-07     	|  20000          	|  0.017798       	|  0.0024319      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.3161e-07     	|  20000          	|  0.016632       	|  0.0022725      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  7.7446e-07     	|  20000          	|  0.015489       	|  0.0021164      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  7.062e-07      	|  20000          	|  0.014124       	|  0.0019298      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  6.9859e-07     	|  20000          	|  0.013972       	|  0.001909       	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  6.8698e-07     	|  20000          	|  0.01374        	|  0.0018773      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.011544       	|  1              	|  0.011544       	|  0.0015773      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0050497      	|  2              	|  0.010099       	|  0.0013799      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00024336     	|  20             	|  0.0048673      	|  0.00066504     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00022872     	|  20             	|  0.0045744      	|  0.00062502     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.517e-05      	|  48             	|  0.0026482      	|  0.00036183     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011071     	|  20             	|  0.0022142      	|  0.00030253     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.0010375      	|  2              	|  0.002075       	|  0.00028351     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0006307      	|  2              	|  0.0012614      	|  0.00017235     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00048013     	|  2              	|  0.00096026     	|  0.0001312      	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00042238     	|  2              	|  0.00084476     	|  0.00011542     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00064318     	|  1              	|  0.00064318     	|  8.7881e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00054496     	|  1              	|  0.00054496     	|  7.4461e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00046567     	|  1              	|  0.00046567     	|  6.3626e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00031598     	|  1              	|  0.00031598     	|  4.3173e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.1524e-06     	|  48             	|  0.00029532     	|  4.035e-05      	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.6572e-06     	|  48             	|  0.00017554     	|  2.3985e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.2662e-06     	|  48             	|  0.00015678     	|  2.1421e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.114e-06      	|  48             	|  0.00014947     	|  2.0423e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8862e-06     	|  48             	|  0.00013854     	|  1.8929e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.5188e-06     	|  20             	|  0.00011038     	|  1.5081e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  4.5705e-05     	|  2              	|  9.1409e-05     	|  1.249e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.5293e-06     	|  20             	|  9.0586e-05     	|  1.2377e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.6086e-06     	|  48             	|  7.7213e-05     	|  1.055e-05      	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.5907e-06     	|  48             	|  7.6355e-05     	|  1.0433e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.0468e-06     	|  22             	|  6.703e-05      	|  9.1586e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2811e-06     	|  48             	|  6.1494e-05     	|  8.4022e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.6714e-06     	|  22             	|  5.877e-05      	|  8.03e-06       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.6549e-06     	|  20             	|  5.3098e-05     	|  7.255e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.8559e-05     	|  1              	|  4.8559e-05     	|  6.6348e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.054e-06      	|  22             	|  4.5189e-05     	|  6.1743e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.0122e-06     	|  22             	|  4.4269e-05     	|  6.0486e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.0023e-06     	|  22             	|  4.405e-05      	|  6.0187e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  1.9781e-06     	|  22             	|  4.3519e-05     	|  5.9461e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.4924e-06     	|  20             	|  2.9849e-05     	|  4.0784e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.2398e-06     	|  22             	|  2.7276e-05     	|  3.7268e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.2281e-06     	|  22             	|  2.7018e-05     	|  3.6915e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.2672e-06     	|  20             	|  2.5343e-05     	|  3.4627e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  7.6946e-06     	|  2              	|  1.5389e-05     	|  2.1027e-06     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.2861e-05     	|  1              	|  1.2861e-05     	|  1.7572e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  6.0401e-06     	|  2              	|  1.208e-05      	|  1.6506e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.031e-05      	|  1              	|  1.031e-05      	|  1.4087e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  9.94e-06       	|  1              	|  9.94e-06       	|  1.3581e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  4.78e-06       	|  2              	|  9.56e-06       	|  1.3062e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  9.1698e-06     	|  1              	|  9.1698e-06     	|  1.2529e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  8.11e-06       	|  1              	|  8.11e-06       	|  1.1081e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.6201e-06     	|  2              	|  7.2401e-06     	|  9.8924e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  7.0306e-06     	|  1              	|  7.0306e-06     	|  9.6061e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  3.4198e-06     	|  2              	|  6.8396e-06     	|  9.3453e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.3649e-06     	|  2              	|  6.7297e-06     	|  9.1951e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  3.1302e-06     	|  2              	|  6.2604e-06     	|  8.5538e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  5.6205e-06     	|  1              	|  5.6205e-06     	|  7.6796e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  5.5898e-06     	|  1              	|  5.5898e-06     	|  7.6376e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  5.5097e-06     	|  1              	|  5.5097e-06     	|  7.5281e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  5.4799e-06     	|  1              	|  5.4799e-06     	|  7.4874e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  4.9407e-06     	|  1              	|  4.9407e-06     	|  6.7506e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  4.7004e-06     	|  1              	|  4.7004e-06     	|  6.4223e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.6901e-06     	|  1              	|  4.6901e-06     	|  6.4083e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.325e-06      	|  2              	|  4.6501e-06     	|  6.3536e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.5802e-06     	|  1              	|  4.5802e-06     	|  6.2582e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  4.2804e-06     	|  1              	|  4.2804e-06     	|  5.8484e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.2599e-06     	|  1              	|  4.2599e-06     	|  5.8204e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.0205e-06     	|  1              	|  4.0205e-06     	|  5.4934e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0098e-06     	|  2              	|  4.0196e-06     	|  5.4921e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.9348e-06     	|  2              	|  3.8696e-06     	|  5.2872e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  1.9148e-06     	|  2              	|  3.8296e-06     	|  5.2325e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.9101e-06     	|  2              	|  3.8203e-06     	|  5.2198e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  3.7597e-06     	|  1              	|  3.7597e-06     	|  5.1371e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.8347e-06     	|  2              	|  3.6694e-06     	|  5.0137e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.54e-06       	|  1              	|  3.54e-06       	|  4.8368e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.4897e-06     	|  1              	|  3.4897e-06     	|  4.7681e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.6154e-06     	|  2              	|  3.2308e-06     	|  4.4143e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.0203e-06     	|  1              	|  3.0203e-06     	|  4.1267e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.55e-06       	|  1              	|  2.55e-06       	|  3.4841e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.5304e-06     	|  1              	|  2.5304e-06     	|  3.4574e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.4503e-06     	|  1              	|  2.4503e-06     	|  3.348e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.4298e-06     	|  1              	|  2.4298e-06     	|  3.32e-07       	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.4205e-06     	|  1              	|  2.4205e-06     	|  3.3072e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.3497e-06     	|  1              	|  2.3497e-06     	|  3.2105e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.2994e-06     	|  1              	|  2.2994e-06     	|  3.1418e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.2696e-06     	|  1              	|  2.2696e-06     	|  3.1011e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.2603e-06     	|  1              	|  2.2603e-06     	|  3.0884e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  2.9242e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.1299e-06     	|  1              	|  2.1299e-06     	|  2.9102e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1094e-06     	|  1              	|  2.1094e-06     	|  2.8822e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.0992e-06     	|  1              	|  2.0992e-06     	|  2.8682e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.9195e-06     	|  1              	|  1.9195e-06     	|  2.6226e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8589e-06     	|  1              	|  1.8589e-06     	|  2.5399e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.5907e-06     	|  1              	|  1.5907e-06     	|  2.1734e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.3905e-06     	|  1              	|  1.3905e-06     	|  1.8998e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.2601e-06     	|  1              	|  1.2601e-06     	|  1.7217e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0394e-06     	|  1              	|  1.0394e-06     	|  1.4201e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
