FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881841         	|  2788.3         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  138.52         	|  20             	|  2770.4         	|  99.358         	|
|  run_training_batch                                                                                                                                                                                         	|  0.13111        	|  20000          	|  2622.2         	|  94.04          	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.13077        	|  20000          	|  2615.5         	|  93.8           	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.031051       	|  20000          	|  621.02         	|  22.272         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.005073       	|  20000          	|  101.46         	|  3.6388         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0031998      	|  20000          	|  63.995         	|  2.2951         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.69514        	|  86             	|  59.782         	|  2.144          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00047763     	|  20000          	|  9.5526         	|  0.34259        	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  7.6572         	|  1              	|  7.6572         	|  0.27461        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.21671        	|  20             	|  4.3342         	|  0.15544        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00013109     	|  20086          	|  2.6331         	|  0.094432       	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  8.3797e-05     	|  20086          	|  1.6831         	|  0.060364       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.5818e-05     	|  20000          	|  0.51636        	|  0.018519       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0057194      	|  86             	|  0.49187        	|  0.01764        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.5876e-05     	|  20000          	|  0.31752        	|  0.011388       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.5227e-05     	|  20000          	|  0.30454        	|  0.010922       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.085203       	|  2              	|  0.17041        	|  0.0061113      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  5.3349e-06     	|  20000          	|  0.1067         	|  0.0038265      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.4961e-06     	|  20000          	|  0.069921       	|  0.0025076      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.2593e-06     	|  20000          	|  0.065186       	|  0.0023378      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.0124e-06     	|  20000          	|  0.060248       	|  0.0021607      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.9194e-06     	|  20000          	|  0.058389       	|  0.002094       	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.8673e-06     	|  20000          	|  0.057347       	|  0.0020567      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.6583e-06     	|  20000          	|  0.053166       	|  0.0019067      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.5301e-06     	|  20000          	|  0.050601       	|  0.0018147      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.468e-06      	|  20000          	|  0.049359       	|  0.0017702      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.3483e-06     	|  20000          	|  0.046967       	|  0.0016844      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2987e-06     	|  20000          	|  0.045974       	|  0.0016488      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.2219e-06     	|  20000          	|  0.044438       	|  0.0015937      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2041e-06     	|  20000          	|  0.044082       	|  0.0015809      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.147e-06      	|  20000          	|  0.042941       	|  0.00154        	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.1226e-06     	|  20000          	|  0.042451       	|  0.0015225      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00049198     	|  86             	|  0.04231        	|  0.0015174      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.1098e-06     	|  20000          	|  0.042196       	|  0.0015133      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0748e-06     	|  20000          	|  0.041496       	|  0.0014882      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.0644e-06     	|  20000          	|  0.041288       	|  0.0014807      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0462e-06     	|  20000          	|  0.040923       	|  0.0014677      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9962e-06     	|  20000          	|  0.039925       	|  0.0014318      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.9808e-06     	|  20086          	|  0.039787       	|  0.0014269      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9651e-06     	|  20000          	|  0.039302       	|  0.0014095      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.9318e-06     	|  20000          	|  0.038636       	|  0.0013856      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.237e-06      	|  20000          	|  0.02474        	|  0.00088727     	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.0928e-06     	|  20000          	|  0.021855       	|  0.00078381     	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.0595e-06     	|  20000          	|  0.021191       	|  0.00075998     	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.0219e-06     	|  20086          	|  0.020526       	|  0.00073615     	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.0105e-06     	|  20000          	|  0.02021        	|  0.00072481     	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  9.8382e-07     	|  20000          	|  0.019676       	|  0.00070567     	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  9.8185e-07     	|  20000          	|  0.019637       	|  0.00070425     	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  9.6619e-07     	|  20000          	|  0.019324       	|  0.00069302     	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  9.1616e-07     	|  20000          	|  0.018323       	|  0.00065714     	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  9.0835e-07     	|  20000          	|  0.018167       	|  0.00065153     	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0038691      	|  2              	|  0.0077381      	|  0.00027752     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.0002359      	|  20             	|  0.004718       	|  0.00016921     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.0002193      	|  20             	|  0.0043859      	|  0.0001573      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0034659      	|  1              	|  0.0034659      	|  0.0001243      	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  2.8121e-05     	|  86             	|  0.0024184      	|  8.6734e-05     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.7342e-05     	|  20             	|  0.0019468      	|  6.9821e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00053466     	|  2              	|  0.0010693      	|  3.835e-05      	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00051192     	|  2              	|  0.0010238      	|  3.6719e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00086428     	|  1              	|  0.00086428     	|  3.0996e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00040193     	|  2              	|  0.00080387     	|  2.883e-05      	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00028075     	|  2              	|  0.00056149     	|  2.0137e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.4354e-06     	|  86             	|  0.00055345     	|  1.9849e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.0002915      	|  1              	|  0.0002915      	|  1.0454e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.3383e-06     	|  86             	|  0.0002871      	|  1.0296e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00025972     	|  1              	|  0.00025972     	|  9.3144e-06     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00025849     	|  1              	|  0.00025849     	|  9.2703e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.9163e-06     	|  86             	|  0.0002508      	|  8.9947e-06     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.7025e-06     	|  86             	|  0.00023241     	|  8.3352e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.5952e-06     	|  86             	|  0.00022319     	|  8.0042e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.0432e-06     	|  86             	|  0.00017572     	|  6.3019e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.5182e-06     	|  86             	|  0.00013056     	|  4.6825e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.4769e-06     	|  20             	|  0.00012954     	|  4.6457e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2217e-06     	|  86             	|  0.00010507     	|  3.7682e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.9834e-06     	|  20             	|  9.9668e-05     	|  3.5745e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.3372e-06     	|  22             	|  7.3418e-05     	|  2.633e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.2614e-05     	|  2              	|  6.5228e-05     	|  2.3393e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.9598e-06     	|  22             	|  6.5116e-05     	|  2.3353e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.837e-06      	|  20             	|  5.674e-05      	|  2.0349e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.1436e-06     	|  22             	|  4.7159e-05     	|  1.6913e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.0749e-06     	|  22             	|  4.5648e-05     	|  1.6371e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.0527e-06     	|  22             	|  4.516e-05      	|  1.6196e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.0481e-06     	|  22             	|  4.5058e-05     	|  1.616e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.8549e-05     	|  1              	|  3.8549e-05     	|  1.3825e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3632e-06     	|  22             	|  2.999e-05      	|  1.0755e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.4905e-06     	|  20             	|  2.981e-05      	|  1.0691e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3195e-06     	|  22             	|  2.9029e-05     	|  1.0411e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.19e-06       	|  20             	|  2.3801e-05     	|  8.5358e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  8.4494e-06     	|  2              	|  1.6899e-05     	|  6.0605e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.6799e-06     	|  2              	|  1.336e-05      	|  4.7913e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  9.8106e-06     	|  1              	|  9.8106e-06     	|  3.5184e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  9.3803e-06     	|  1              	|  9.3803e-06     	|  3.3641e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  8.109e-06      	|  1              	|  8.109e-06      	|  2.9082e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  8.0904e-06     	|  1              	|  8.0904e-06     	|  2.9015e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  8.0494e-06     	|  1              	|  8.0494e-06     	|  2.8868e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.8147e-06     	|  2              	|  7.6294e-06     	|  2.7362e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.635e-06      	|  2              	|  7.2699e-06     	|  2.6072e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  3.4207e-06     	|  2              	|  6.8415e-06     	|  2.4536e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.1749e-06     	|  2              	|  6.3498e-06     	|  2.2773e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.1497e-06     	|  2              	|  6.2995e-06     	|  2.2592e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  6.2697e-06     	|  1              	|  6.2697e-06     	|  2.2485e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  6.13e-06       	|  1              	|  6.13e-06       	|  2.1984e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.0499e-06     	|  1              	|  6.0499e-06     	|  2.1697e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.5854e-06     	|  2              	|  5.1707e-06     	|  1.8544e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  5.1595e-06     	|  1              	|  5.1595e-06     	|  1.8504e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  5.1595e-06     	|  1              	|  5.1595e-06     	|  1.8504e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.5546e-06     	|  2              	|  5.1092e-06     	|  1.8324e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.4955e-06     	|  2              	|  4.991e-06      	|  1.7899e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.9202e-06     	|  1              	|  4.9202e-06     	|  1.7646e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  4.9006e-06     	|  1              	|  4.9006e-06     	|  1.7575e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  4.8606e-06     	|  1              	|  4.8606e-06     	|  1.7432e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  4.6305e-06     	|  1              	|  4.6305e-06     	|  1.6607e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  4.6203e-06     	|  1              	|  4.6203e-06     	|  1.657e-07      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.2803e-06     	|  2              	|  4.5607e-06     	|  1.6356e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  4.38e-06       	|  1              	|  4.38e-06       	|  1.5708e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0852e-06     	|  2              	|  4.1705e-06     	|  1.4957e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.1602e-06     	|  1              	|  4.1602e-06     	|  1.492e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.02e-06       	|  2              	|  4.0401e-06     	|  1.4489e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  3.8696e-06     	|  1              	|  3.8696e-06     	|  1.3878e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.6992e-06     	|  1              	|  3.6992e-06     	|  1.3267e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.5297e-06     	|  1              	|  3.5297e-06     	|  1.2659e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.2196e-06     	|  1              	|  3.2196e-06     	|  1.1547e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  3.2e-06        	|  1              	|  3.2e-06        	|  1.1476e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.0501e-06     	|  1              	|  3.0501e-06     	|  1.0939e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  2.9895e-06     	|  1              	|  2.9895e-06     	|  1.0722e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.6198e-06     	|  1              	|  2.6198e-06     	|  9.3956e-08     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.5099e-06     	|  1              	|  2.5099e-06     	|  9.0015e-08     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.5006e-06     	|  1              	|  2.5006e-06     	|  8.9681e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.4484e-06     	|  1              	|  2.4484e-06     	|  8.781e-08      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.4e-06        	|  1              	|  2.4e-06        	|  8.6073e-08     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.1446e-06     	|  2              	|  2.2892e-06     	|  8.2099e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.2696e-06     	|  1              	|  2.2696e-06     	|  8.1397e-08     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.2398e-06     	|  1              	|  2.2398e-06     	|  8.0328e-08     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.21e-06       	|  1              	|  2.21e-06       	|  7.926e-08      	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  2.1607e-06     	|  1              	|  2.1607e-06     	|  7.7489e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0405e-06     	|  1              	|  2.0405e-06     	|  7.3181e-08     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.6103e-06     	|  1              	|  1.6103e-06     	|  5.775e-08      	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.6e-06        	|  1              	|  1.6e-06        	|  5.7382e-08     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.5693e-06     	|  1              	|  1.5693e-06     	|  5.628e-08      	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.1707e-06     	|  1              	|  1.1707e-06     	|  4.1985e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1604e-06     	|  1              	|  1.1604e-06     	|  4.1617e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0105e-06     	|  1              	|  1.0105e-06     	|  3.624e-08      	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
