FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881265         	|  1045.8         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  50.786         	|  20             	|  1015.7         	|  97.12          	|
|  run_training_batch                                                                                                                                                                                         	|  0.044247       	|  20000          	|  884.94         	|  84.615         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.043989       	|  20000          	|  879.78         	|  84.121         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.011778       	|  20000          	|  235.57         	|  22.524         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0036136      	|  20000          	|  72.273         	|  6.9105         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.001933       	|  20000          	|  38.659         	|  3.6965         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.69725        	|  50             	|  34.862         	|  3.3334         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  14.05          	|  1              	|  14.05          	|  1.3434         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00047428     	|  20000          	|  9.4855         	|  0.90697        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.26558        	|  20             	|  5.3116         	|  0.50788        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00012767     	|  20050          	|  2.5599         	|  0.24477        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  8.1135e-05     	|  20050          	|  1.6268         	|  0.15555        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0083416      	|  50             	|  0.41708        	|  0.03988        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.0749e-05     	|  20000          	|  0.41498        	|  0.039679       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.4957e-05     	|  20000          	|  0.29914        	|  0.028602       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.468e-05      	|  20000          	|  0.2936         	|  0.028073       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.364e-06      	|  20000          	|  0.087281       	|  0.0083455      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.2591e-06     	|  20000          	|  0.085181       	|  0.0081447      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.9087e-06     	|  20000          	|  0.078175       	|  0.0074748      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.035785       	|  2              	|  0.07157        	|  0.0068433      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.5263e-06     	|  20000          	|  0.070526       	|  0.0067434      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.4012e-06     	|  20000          	|  0.068024       	|  0.0065042      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.1871e-06     	|  20000          	|  0.063742       	|  0.0060948      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.8619e-06     	|  20000          	|  0.057239       	|  0.0054729      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.8616e-06     	|  20000          	|  0.057232       	|  0.0054723      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.62e-06       	|  20000          	|  0.052399       	|  0.0050102      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.4361e-06     	|  20000          	|  0.048722       	|  0.0046586      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.3828e-06     	|  20000          	|  0.047657       	|  0.0045568      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.3405e-06     	|  20000          	|  0.046811       	|  0.0044759      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3312e-06     	|  20000          	|  0.046625       	|  0.0044581      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.3269e-06     	|  20000          	|  0.046537       	|  0.0044497      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2244e-06     	|  20000          	|  0.044488       	|  0.0042538      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.1984e-06     	|  20000          	|  0.043969       	|  0.0042042      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.187e-06      	|  20000          	|  0.043739       	|  0.0041822      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1373e-06     	|  20000          	|  0.042746       	|  0.0040872      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1099e-06     	|  20000          	|  0.042198       	|  0.0040349      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0942e-06     	|  20000          	|  0.041883       	|  0.0040048      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.0712e-06     	|  20000          	|  0.041424       	|  0.0039608      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0604e-06     	|  20000          	|  0.041208       	|  0.0039401      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.53e-06       	|  20000          	|  0.030599       	|  0.0029258      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.4956e-06     	|  20050          	|  0.029988       	|  0.0028673      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00045686     	|  50             	|  0.022843       	|  0.0021842      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.0699e-06     	|  20000          	|  0.021397       	|  0.0020459      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.0322e-06     	|  20050          	|  0.020696       	|  0.0019788      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.027e-06      	|  20000          	|  0.020541       	|  0.001964       	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  9.9057e-07     	|  20000          	|  0.019811       	|  0.0018943      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.9392e-07     	|  20000          	|  0.017878       	|  0.0017095      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.9008e-07     	|  20000          	|  0.017802       	|  0.0017021      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  8.787e-07      	|  20000          	|  0.017574       	|  0.0016804      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  8.5779e-07     	|  20000          	|  0.017156       	|  0.0016404      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  8.3171e-07     	|  20000          	|  0.016634       	|  0.0015905      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0046711      	|  2              	|  0.0093421      	|  0.00089326     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0055924      	|  1              	|  0.0055924      	|  0.00053473     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00022468     	|  20             	|  0.0044935      	|  0.00042965     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00021778     	|  20             	|  0.0043557      	|  0.00041648     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.0527e-05     	|  50             	|  0.0025263      	|  0.00024156     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.3819e-05     	|  20             	|  0.0018764      	|  0.00017941     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00058279     	|  2              	|  0.0011656      	|  0.00011145     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00042661     	|  2              	|  0.00085322     	|  8.1582e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00039502     	|  2              	|  0.00079003     	|  7.554e-05      	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00065836     	|  1              	|  0.00065836     	|  6.295e-05      	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00021454     	|  2              	|  0.00042909     	|  4.1028e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00033497     	|  1              	|  0.00033497     	|  3.2028e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.1457e-06     	|  50             	|  0.00030728     	|  2.9381e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00027325     	|  1              	|  0.00027325     	|  2.6127e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00026932     	|  1              	|  0.00026932     	|  2.5751e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.4038e-06     	|  50             	|  0.00022019     	|  2.1054e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.2737e-06     	|  50             	|  0.00016369     	|  1.5651e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.0809e-06     	|  50             	|  0.00015404     	|  1.4729e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8758e-06     	|  50             	|  0.00014379     	|  1.3749e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.7859e-06     	|  20             	|  0.00011572     	|  1.1065e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  5.5305e-06     	|  20             	|  0.00011061     	|  1.0576e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.0013e-06     	|  50             	|  0.00010007     	|  9.568e-06      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.315e-06      	|  20             	|  8.63e-05       	|  8.2517e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.5527e-06     	|  22             	|  7.8159e-05     	|  7.4733e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.4586e-06     	|  22             	|  7.6089e-05     	|  7.2754e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.502e-06      	|  50             	|  7.51e-05       	|  7.1808e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2219e-06     	|  50             	|  6.1094e-05     	|  5.8416e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  2.904e-05      	|  2              	|  5.808e-05      	|  5.5534e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.1445e-06     	|  22             	|  4.718e-05      	|  4.5112e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1308e-06     	|  22             	|  4.6878e-05     	|  4.4823e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.1269e-06     	|  22             	|  4.6792e-05     	|  4.474e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.0844e-06     	|  22             	|  4.5857e-05     	|  4.3847e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.1909e-05     	|  1              	|  3.1909e-05     	|  3.051e-06      	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3968e-06     	|  22             	|  3.0729e-05     	|  2.9382e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.4345e-05     	|  2              	|  2.869e-05      	|  2.7433e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.4075e-06     	|  20             	|  2.8149e-05     	|  2.6915e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.2655e-06     	|  22             	|  2.784e-05      	|  2.662e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.2004e-06     	|  20             	|  2.4008e-05     	|  2.2955e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  5.0701e-06     	|  2              	|  1.014e-05      	|  9.6957e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.9542e-06     	|  2              	|  9.9083e-06     	|  9.474e-07      	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  8.9696e-06     	|  1              	|  8.9696e-06     	|  8.5764e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.4452e-06     	|  2              	|  8.8904e-06     	|  8.5007e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  8.6995e-06     	|  1              	|  8.6995e-06     	|  8.3181e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  7.7998e-06     	|  1              	|  7.7998e-06     	|  7.4579e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.8003e-06     	|  2              	|  7.6005e-06     	|  7.2674e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  7.2587e-06     	|  1              	|  7.2587e-06     	|  6.9405e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.4254e-06     	|  2              	|  6.8508e-06     	|  6.5505e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  6.8098e-06     	|  1              	|  6.8098e-06     	|  6.5113e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.3551e-06     	|  2              	|  6.7102e-06     	|  6.416e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.2051e-06     	|  2              	|  6.4103e-06     	|  6.1293e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.4e-06        	|  1              	|  6.4e-06        	|  6.1195e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  5.8301e-06     	|  1              	|  5.8301e-06     	|  5.5745e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  5.79e-06       	|  1              	|  5.79e-06       	|  5.5362e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.8303e-06     	|  2              	|  5.6606e-06     	|  5.4124e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.7553e-06     	|  2              	|  5.5106e-06     	|  5.2691e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.8298e-06     	|  1              	|  4.8298e-06     	|  4.6181e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  4.7497e-06     	|  1              	|  4.7497e-06     	|  4.5415e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  4.5095e-06     	|  1              	|  4.5095e-06     	|  4.3118e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.1001e-06     	|  2              	|  4.2003e-06     	|  4.0161e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.1695e-06     	|  1              	|  4.1695e-06     	|  3.9868e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.075e-06      	|  2              	|  4.15e-06       	|  3.9681e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.0699e-06     	|  1              	|  4.0699e-06     	|  3.8915e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.9702e-06     	|  1              	|  3.9702e-06     	|  3.7962e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.8007e-06     	|  1              	|  3.8007e-06     	|  3.6341e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.7299e-06     	|  1              	|  3.7299e-06     	|  3.5664e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.5297e-06     	|  1              	|  3.5297e-06     	|  3.375e-07      	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.97e-06       	|  1              	|  2.97e-06       	|  2.8398e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.9495e-06     	|  1              	|  2.9495e-06     	|  2.8202e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3849e-06     	|  2              	|  2.7698e-06     	|  2.6483e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.6301e-06     	|  1              	|  2.6301e-06     	|  2.5148e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.3209e-06     	|  1              	|  2.3209e-06     	|  2.2191e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.1153e-06     	|  2              	|  2.2305e-06     	|  2.1327e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.2007e-06     	|  1              	|  2.2007e-06     	|  2.1042e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.17e-06       	|  1              	|  2.17e-06       	|  2.0749e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.1504e-06     	|  1              	|  2.1504e-06     	|  2.0562e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1495e-06     	|  1              	|  2.1495e-06     	|  2.0553e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.1495e-06     	|  1              	|  2.1495e-06     	|  2.0553e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1299e-06     	|  1              	|  2.1299e-06     	|  2.0366e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.0703e-06     	|  1              	|  2.0703e-06     	|  1.9796e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.0694e-06     	|  1              	|  2.0694e-06     	|  1.9787e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.0405e-06     	|  1              	|  2.0405e-06     	|  1.9511e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.9707e-06     	|  1              	|  1.9707e-06     	|  1.8843e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.9297e-06     	|  1              	|  1.9297e-06     	|  1.8451e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.9204e-06     	|  1              	|  1.9204e-06     	|  1.8362e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8803e-06     	|  1              	|  1.8803e-06     	|  1.7979e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.8496e-06     	|  1              	|  1.8496e-06     	|  1.7685e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.6894e-06     	|  1              	|  1.6894e-06     	|  1.6154e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1902e-06     	|  1              	|  1.1902e-06     	|  1.1381e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.1399e-06     	|  1              	|  1.1399e-06     	|  1.09e-07       	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0496e-06     	|  1              	|  1.0496e-06     	|  1.0036e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.0207e-06     	|  1              	|  1.0207e-06     	|  9.7599e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
