FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881073         	|  929.47         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  44.69          	|  20             	|  893.8          	|  96.163         	|
|  run_training_batch                                                                                                                                                                                         	|  0.032475       	|  20000          	|  649.5          	|  69.878         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.032094       	|  20000          	|  641.88         	|  69.059         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.011973       	|  20000          	|  239.46         	|  25.763         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.009683       	|  20000          	|  193.66         	|  20.835         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.002248       	|  20000          	|  44.959         	|  4.8371         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  19.492         	|  1              	|  19.492         	|  2.0971         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.44149        	|  38             	|  16.777         	|  1.805          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.0006799      	|  20000          	|  13.598         	|  1.463          	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00020062     	|  20038          	|  4.02           	|  0.43251        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.18843        	|  20             	|  3.7686         	|  0.40546        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00012844     	|  20038          	|  2.5736         	|  0.27689        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.2813e-05     	|  20000          	|  0.65627        	|  0.070607       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.2996e-05     	|  20000          	|  0.45992        	|  0.049482       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.0745e-05     	|  20000          	|  0.41489        	|  0.044637       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0078729      	|  38             	|  0.29917        	|  0.032187       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.072338       	|  2              	|  0.14468        	|  0.015565       	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.0784e-06     	|  20000          	|  0.10157        	|  0.010928       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.9451e-06     	|  20000          	|  0.098903       	|  0.010641       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.7621e-06     	|  20000          	|  0.095242       	|  0.010247       	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  4.3002e-06     	|  20038          	|  0.086168       	|  0.0092707      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.8518e-06     	|  20000          	|  0.077036       	|  0.0082882      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.8422e-06     	|  20000          	|  0.076843       	|  0.0082674      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.7109e-06     	|  20000          	|  0.074219       	|  0.0079851      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.5398e-06     	|  20000          	|  0.070796       	|  0.0076168      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  3.4376e-06     	|  20000          	|  0.068752       	|  0.0073969      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.2383e-06     	|  20000          	|  0.064765       	|  0.006968       	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  3.0891e-06     	|  20000          	|  0.061781       	|  0.0066469      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  3.0032e-06     	|  20000          	|  0.060065       	|  0.0064622      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.7606e-06     	|  20000          	|  0.055213       	|  0.0059402      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.7181e-06     	|  20000          	|  0.054363       	|  0.0058488      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.6424e-06     	|  20000          	|  0.052849       	|  0.0056859      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.5266e-06     	|  20000          	|  0.050532       	|  0.0054367      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.4285e-06     	|  20000          	|  0.04857        	|  0.0052256      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.428e-06      	|  20000          	|  0.048561       	|  0.0052246      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.3591e-06     	|  20000          	|  0.047183       	|  0.0050763      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.3429e-06     	|  20000          	|  0.046858       	|  0.0050414      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.3046e-06     	|  20000          	|  0.046092       	|  0.0049589      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.2678e-06     	|  20000          	|  0.045356       	|  0.0048798      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.2389e-06     	|  20000          	|  0.044777       	|  0.0048175      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.9605e-06     	|  20000          	|  0.039211       	|  0.0042186      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.9086e-06     	|  20038          	|  0.038244       	|  0.0041146      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.4925e-06     	|  20000          	|  0.02985        	|  0.0032115      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.4614e-06     	|  20000          	|  0.029229       	|  0.0031447      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.457e-06      	|  20000          	|  0.029139       	|  0.003135       	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2644e-06     	|  20000          	|  0.025289       	|  0.0027208      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.2224e-06     	|  20000          	|  0.024448       	|  0.0026303      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1922e-06     	|  20000          	|  0.023844       	|  0.0025653      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.1815e-06     	|  20000          	|  0.02363        	|  0.0025423      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.1113e-06     	|  20000          	|  0.022226       	|  0.0023913      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00045957     	|  38             	|  0.017464       	|  0.0018789      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0048107      	|  2              	|  0.0096213      	|  0.0010351      	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00029261     	|  20             	|  0.0058522      	|  0.00062963     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00025418     	|  20             	|  0.0050836      	|  0.00054693     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0038626      	|  1              	|  0.0038626      	|  0.00041557     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.7931e-05     	|  20             	|  0.0019586      	|  0.00021072     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  4.6736e-05     	|  38             	|  0.001776       	|  0.00019107     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00045991     	|  2              	|  0.00091981     	|  9.8961e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00045715     	|  2              	|  0.00091429     	|  9.8367e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00042362     	|  2              	|  0.00084723     	|  9.1152e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00060794     	|  1              	|  0.00060794     	|  6.5408e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00029573     	|  2              	|  0.00059147     	|  6.3635e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00027905     	|  1              	|  0.00027905     	|  3.0022e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00027887     	|  1              	|  0.00027887     	|  3.0003e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.0002657      	|  1              	|  0.0002657      	|  2.8586e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  5.7952e-06     	|  38             	|  0.00022022     	|  2.3693e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.5113e-06     	|  38             	|  0.00013343     	|  1.4355e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.0755e-06     	|  20             	|  0.00012151     	|  1.3073e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.0605e-06     	|  38             	|  0.0001163      	|  1.2512e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.9464e-06     	|  38             	|  0.00011196     	|  1.2046e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.1153e-06     	|  20             	|  0.00010231     	|  1.1007e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.6608e-06     	|  38             	|  0.00010111     	|  1.0878e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.6818e-06     	|  22             	|  8.1e-05        	|  8.7146e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.6729e-05     	|  2              	|  7.3459e-05     	|  7.9033e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.2957e-06     	|  22             	|  7.2505e-05     	|  7.8007e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7563e-06     	|  38             	|  6.674e-05      	|  7.1805e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.7433e-06     	|  38             	|  6.6247e-05     	|  7.1274e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.017e-06      	|  20             	|  6.0339e-05     	|  6.4918e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  2.4017e-06     	|  22             	|  5.2837e-05     	|  5.6846e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.3776e-06     	|  38             	|  5.2351e-05     	|  5.6323e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.2939e-06     	|  22             	|  5.0467e-05     	|  5.4296e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.2868e-06     	|  22             	|  5.031e-05      	|  5.4128e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2564e-06     	|  22             	|  4.9641e-05     	|  5.3408e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.24e-06       	|  22             	|  4.928e-05      	|  5.3019e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.4221e-05     	|  1              	|  3.4221e-05     	|  3.6817e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.6806e-06     	|  20             	|  3.3611e-05     	|  3.6162e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.4269e-06     	|  22             	|  3.1391e-05     	|  3.3773e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.5206e-06     	|  20             	|  3.0411e-05     	|  3.2719e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  7.7649e-06     	|  2              	|  1.553e-05      	|  1.6708e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  6.9402e-06     	|  2              	|  1.388e-05      	|  1.4934e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.9956e-06     	|  2              	|  9.9912e-06     	|  1.0749e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.645e-06      	|  2              	|  9.2899e-06     	|  9.9949e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  8.5486e-06     	|  1              	|  8.5486e-06     	|  9.1973e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  8.2385e-06     	|  1              	|  8.2385e-06     	|  8.8636e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.8752e-06     	|  2              	|  7.7505e-06     	|  8.3386e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  6.8592e-06     	|  1              	|  6.8592e-06     	|  7.3797e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.6794e-06     	|  1              	|  6.6794e-06     	|  7.1863e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  6.6701e-06     	|  1              	|  6.6701e-06     	|  7.1763e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.2401e-06     	|  2              	|  6.4801e-06     	|  6.9719e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.2247e-06     	|  2              	|  6.4494e-06     	|  6.9388e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.1646e-06     	|  2              	|  6.3293e-06     	|  6.8095e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.1505e-06     	|  1              	|  6.1505e-06     	|  6.6172e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.8103e-06     	|  1              	|  4.8103e-06     	|  5.1753e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.3898e-06     	|  2              	|  4.7795e-06     	|  5.1422e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.3795e-06     	|  2              	|  4.7591e-06     	|  5.1202e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.3353e-06     	|  2              	|  4.6706e-06     	|  5.025e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.3348e-06     	|  2              	|  4.6697e-06     	|  5.024e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.2501e-06     	|  2              	|  4.5002e-06     	|  4.8416e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  4.4489e-06     	|  1              	|  4.4489e-06     	|  4.7865e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.4201e-06     	|  1              	|  4.4201e-06     	|  4.7555e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  4.15e-06       	|  1              	|  4.15e-06       	|  4.4649e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.0205e-06     	|  1              	|  4.0205e-06     	|  4.3256e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.8296e-06     	|  1              	|  3.8296e-06     	|  4.1202e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.7495e-06     	|  1              	|  3.7495e-06     	|  4.034e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  3.6703e-06     	|  1              	|  3.6703e-06     	|  3.9489e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.3593e-06     	|  1              	|  3.3593e-06     	|  3.6142e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.2e-06        	|  1              	|  3.2e-06        	|  3.4428e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  3.0501e-06     	|  1              	|  3.0501e-06     	|  3.2815e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  2.9998e-06     	|  1              	|  2.9998e-06     	|  3.2274e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.9504e-06     	|  1              	|  2.9504e-06     	|  3.1743e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.8498e-06     	|  1              	|  2.8498e-06     	|  3.0661e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.4198e-06     	|  2              	|  2.8396e-06     	|  3.0551e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.78e-06       	|  1              	|  2.78e-06       	|  2.991e-07      	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.6906e-06     	|  1              	|  2.6906e-06     	|  2.8948e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.6599e-06     	|  1              	|  2.6599e-06     	|  2.8617e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.6198e-06     	|  1              	|  2.6198e-06     	|  2.8186e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  2.5602e-06     	|  1              	|  2.5602e-06     	|  2.7545e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.5108e-06     	|  1              	|  2.5108e-06     	|  2.7014e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.4997e-06     	|  1              	|  2.4997e-06     	|  2.6894e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.3795e-06     	|  1              	|  2.3795e-06     	|  2.5601e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.3693e-06     	|  1              	|  2.3693e-06     	|  2.5491e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.3106e-06     	|  1              	|  2.3106e-06     	|  2.4859e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.1607e-06     	|  1              	|  2.1607e-06     	|  2.3246e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0303e-06     	|  1              	|  2.0303e-06     	|  2.1843e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.0098e-06     	|  1              	|  2.0098e-06     	|  2.1623e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.75e-06       	|  1              	|  1.75e-06       	|  1.8827e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.7202e-06     	|  1              	|  1.7202e-06     	|  1.8507e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.6e-06        	|  1              	|  1.6e-06        	|  1.7214e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.2498e-06     	|  1              	|  1.2498e-06     	|  1.3447e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.22e-06       	|  1              	|  1.22e-06       	|  1.3126e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.1697e-06     	|  1              	|  1.1697e-06     	|  1.2585e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  9.3039e-07     	|  1              	|  9.3039e-07     	|  1.001e-07      	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
