FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881297         	|  1542.3         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  75.0           	|  20             	|  1500.0         	|  97.258         	|
|  run_training_batch                                                                                                                                                                                         	|  0.06513        	|  20000          	|  1302.6         	|  84.459         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.064764       	|  20000          	|  1295.3         	|  83.985         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.019218       	|  20000          	|  384.37         	|  24.922         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0066834      	|  20000          	|  133.67         	|  8.6669         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0022007      	|  20000          	|  44.014         	|  2.8538         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.59206        	|  52             	|  30.787         	|  1.9962         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  27.69          	|  1              	|  27.69          	|  1.7954         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.0006535      	|  20000          	|  13.07          	|  0.84745        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00022718     	|  20052          	|  4.5554         	|  0.29537        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.18599        	|  20             	|  3.7198         	|  0.24119        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00014442     	|  20052          	|  2.8959         	|  0.18777        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.1462e-05     	|  20000          	|  0.62924        	|  0.0408         	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.0892e-05     	|  20000          	|  0.41784        	|  0.027093       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.8668e-05     	|  20000          	|  0.37337        	|  0.024209       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0062371      	|  52             	|  0.32433        	|  0.021029       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  5.7943e-06     	|  20000          	|  0.11589        	|  0.007514       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.047329       	|  2              	|  0.094657       	|  0.0061375      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.9731e-06     	|  20000          	|  0.079462       	|  0.0051523      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.9641e-06     	|  20000          	|  0.079281       	|  0.0051405      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.8551e-06     	|  20000          	|  0.077103       	|  0.0049993      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.7779e-06     	|  20000          	|  0.075558       	|  0.0048991      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.7099e-06     	|  20000          	|  0.074198       	|  0.004811       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  3.681e-06      	|  20000          	|  0.073621       	|  0.0047735      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.61e-06       	|  20000          	|  0.072201       	|  0.0046814      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.3784e-06     	|  20052          	|  0.067744       	|  0.0043924      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  3.2661e-06     	|  20000          	|  0.065321       	|  0.0042354      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.106e-06      	|  20000          	|  0.062121       	|  0.0040279      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.0577e-06     	|  20000          	|  0.061154       	|  0.0039652      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.5725e-06     	|  20000          	|  0.051451       	|  0.003336       	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.53e-06       	|  20000          	|  0.0506         	|  0.0032809      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.4638e-06     	|  20000          	|  0.049276       	|  0.003195       	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.4381e-06     	|  20000          	|  0.048761       	|  0.0031616      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.3543e-06     	|  20000          	|  0.047087       	|  0.0030531      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.2336e-06     	|  20000          	|  0.044671       	|  0.0028964      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.2207e-06     	|  20000          	|  0.044415       	|  0.0028798      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.1861e-06     	|  20000          	|  0.043721       	|  0.0028348      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1253e-06     	|  20000          	|  0.042507       	|  0.0027561      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.1036e-06     	|  20000          	|  0.042072       	|  0.0027279      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0937e-06     	|  20000          	|  0.041874       	|  0.0027151      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.6885e-06     	|  20000          	|  0.03377        	|  0.0021896      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.5013e-06     	|  20052          	|  0.030104       	|  0.0019519      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2907e-06     	|  20000          	|  0.025814       	|  0.0016737      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.2531e-06     	|  20000          	|  0.025061       	|  0.0016249      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.2263e-06     	|  20000          	|  0.024525       	|  0.0015902      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004625      	|  52             	|  0.02405        	|  0.0015594      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.141e-06      	|  20000          	|  0.02282        	|  0.0014796      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.1039e-06     	|  20000          	|  0.022077       	|  0.0014315      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.0987e-06     	|  20000          	|  0.021973       	|  0.0014247      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.0955e-06     	|  20000          	|  0.021909       	|  0.0014206      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.0235e-06     	|  20000          	|  0.020471       	|  0.0013273      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0048432      	|  2              	|  0.0096864      	|  0.00062806     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00032262     	|  20             	|  0.0064523      	|  0.00041836     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  0.00010632     	|  52             	|  0.0055284      	|  0.00035846     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00025241     	|  20             	|  0.0050481      	|  0.00032732     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0033892      	|  1              	|  0.0033892      	|  0.00021976     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00010445     	|  20             	|  0.002089       	|  0.00013545     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.0013952      	|  1              	|  0.0013952      	|  9.0463e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00056169     	|  2              	|  0.0011234      	|  7.2839e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00048704     	|  2              	|  0.00097408     	|  6.3159e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00041394     	|  2              	|  0.00082787     	|  5.3679e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00032692     	|  2              	|  0.00065383     	|  4.2394e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  9.0811e-06     	|  52             	|  0.00047222     	|  3.0618e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.000426       	|  1              	|  0.000426       	|  2.7621e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.0004153      	|  1              	|  0.0004153      	|  2.6928e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.0002419      	|  1              	|  0.0002419      	|  1.5684e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.288e-06      	|  52             	|  0.00022298     	|  1.4458e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.2539e-06     	|  52             	|  0.0001692      	|  1.0971e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  3.194e-06      	|  52             	|  0.00016609     	|  1.0769e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.8448e-06     	|  52             	|  0.00014793     	|  9.5916e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.0939e-06     	|  20             	|  0.00012188     	|  7.9024e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.2794e-06     	|  20             	|  0.00010559     	|  6.8462e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  4.677e-05      	|  2              	|  9.3539e-05     	|  6.065e-06      	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7446e-06     	|  52             	|  9.0719e-05     	|  5.8822e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.5804e-06     	|  52             	|  8.2182e-05     	|  5.3286e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.3332e-06     	|  22             	|  7.333e-05      	|  4.7547e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.3708e-06     	|  52             	|  7.1282e-05     	|  4.6219e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.1651e-06     	|  22             	|  6.9632e-05     	|  4.5149e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.0234e-06     	|  20             	|  6.0467e-05     	|  3.9206e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1932e-06     	|  22             	|  4.8251e-05     	|  3.1285e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1563e-06     	|  22             	|  4.7438e-05     	|  3.0758e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.1313e-06     	|  22             	|  4.6889e-05     	|  3.0403e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.0918e-06     	|  22             	|  4.602e-05      	|  2.9839e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.2821e-05     	|  1              	|  3.2821e-05     	|  2.1281e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.6378e-06     	|  20             	|  3.2756e-05     	|  2.1239e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5894e-06     	|  20             	|  3.1789e-05     	|  2.0612e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.4219e-06     	|  22             	|  3.1281e-05     	|  2.0283e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.409e-06      	|  22             	|  3.0999e-05     	|  2.01e-06       	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.4059e-05     	|  1              	|  2.4059e-05     	|  1.56e-06       	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.063e-05      	|  2              	|  2.1259e-05     	|  1.3784e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  7.5847e-06     	|  2              	|  1.5169e-05     	|  9.8357e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.53e-06       	|  2              	|  1.306e-05      	|  8.468e-07      	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  6.1044e-06     	|  2              	|  1.2209e-05     	|  7.916e-07      	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  1.2009e-05     	|  1              	|  1.2009e-05     	|  7.7868e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.4496e-06     	|  2              	|  1.0899e-05     	|  7.067e-07      	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  8.0196e-06     	|  1              	|  8.0196e-06     	|  5.1999e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  7.7598e-06     	|  1              	|  7.7598e-06     	|  5.0314e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.7053e-06     	|  2              	|  7.4105e-06     	|  4.8049e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.6354e-06     	|  2              	|  7.2708e-06     	|  4.7144e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  7.1703e-06     	|  1              	|  7.1703e-06     	|  4.6491e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.2294e-06     	|  2              	|  6.4587e-06     	|  4.1878e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.9849e-06     	|  2              	|  5.9698e-06     	|  3.8708e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  5.4995e-06     	|  1              	|  5.4995e-06     	|  3.5658e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.8904e-06     	|  1              	|  4.8904e-06     	|  3.1709e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.3399e-06     	|  2              	|  4.6799e-06     	|  3.0344e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  4.6501e-06     	|  1              	|  4.6501e-06     	|  3.0151e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.2901e-06     	|  2              	|  4.5802e-06     	|  2.9698e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.4806e-06     	|  1              	|  4.4806e-06     	|  2.9052e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.4499e-06     	|  1              	|  4.4499e-06     	|  2.8853e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  4.4201e-06     	|  1              	|  4.4201e-06     	|  2.8659e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  4.3903e-06     	|  1              	|  4.3903e-06     	|  2.8466e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.3297e-06     	|  1              	|  4.3297e-06     	|  2.8074e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.0801e-06     	|  1              	|  4.0801e-06     	|  2.6455e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  3.8203e-06     	|  1              	|  3.8203e-06     	|  2.477e-07      	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.845e-06      	|  2              	|  3.6899e-06     	|  2.3925e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.5902e-06     	|  1              	|  3.5902e-06     	|  2.3279e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.5008e-06     	|  1              	|  3.5008e-06     	|  2.2699e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.2699e-06     	|  1              	|  3.2699e-06     	|  2.1202e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.2e-06        	|  1              	|  3.2e-06        	|  2.0749e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.5348e-06     	|  2              	|  3.0696e-06     	|  1.9903e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.8601e-06     	|  1              	|  2.8601e-06     	|  1.8545e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.59e-06       	|  1              	|  2.59e-06       	|  1.6793e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.5798e-06     	|  1              	|  2.5798e-06     	|  1.6727e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.4401e-06     	|  1              	|  2.4401e-06     	|  1.5821e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.36e-06       	|  1              	|  2.36e-06       	|  1.5302e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.2994e-06     	|  1              	|  2.2994e-06     	|  1.4909e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.2398e-06     	|  1              	|  2.2398e-06     	|  1.4523e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.2296e-06     	|  1              	|  2.2296e-06     	|  1.4456e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.21e-06       	|  1              	|  2.21e-06       	|  1.433e-07      	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.0952e-06     	|  2              	|  2.1905e-06     	|  1.4203e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1793e-06     	|  1              	|  2.1793e-06     	|  1.413e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.1104e-06     	|  1              	|  2.1104e-06     	|  1.3684e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.9697e-06     	|  1              	|  1.9697e-06     	|  1.2772e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8999e-06     	|  1              	|  1.8999e-06     	|  1.2319e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8701e-06     	|  1              	|  1.8701e-06     	|  1.2126e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.7602e-06     	|  1              	|  1.7602e-06     	|  1.1413e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.7202e-06     	|  1              	|  1.7202e-06     	|  1.1153e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.5004e-06     	|  1              	|  1.5004e-06     	|  9.7282e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.3299e-06     	|  1              	|  1.3299e-06     	|  8.6232e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.1204e-06     	|  1              	|  1.1204e-06     	|  7.2645e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  9.9931e-07     	|  1              	|  9.9931e-07     	|  6.4794e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  9.099e-07      	|  1              	|  9.099e-07      	|  5.8997e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
