FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  885425         	|  1993.7         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  90.338         	|  20             	|  1806.8         	|  90.623         	|
|  run_training_batch                                                                                                                                                                                         	|  0.071348       	|  20000          	|  1427.0         	|  71.574         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.07103        	|  20000          	|  1420.6         	|  71.254         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.018539       	|  20000          	|  370.79         	|  18.598         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.71828        	|  310            	|  222.67         	|  11.169         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  176.15         	|  1              	|  176.15         	|  8.8352         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0061549      	|  20000          	|  123.1          	|  6.1744         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0020759      	|  20000          	|  41.518         	|  2.0825         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00064619     	|  20000          	|  12.924         	|  0.64823        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00017517     	|  20310          	|  3.5577         	|  0.17845        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.15663        	|  20             	|  3.1326         	|  0.15712        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00010832     	|  20310          	|  2.1999         	|  0.11034        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0028118      	|  310            	|  0.87166        	|  0.043721       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.4855e-05     	|  20000          	|  0.49709        	|  0.024933       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.9706e-05     	|  20000          	|  0.39412        	|  0.019768       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.8839e-05     	|  20000          	|  0.37679        	|  0.018899       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004793      	|  310            	|  0.14858        	|  0.0074527      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  6.2701e-06     	|  20000          	|  0.1254         	|  0.0062899      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  5.2062e-06     	|  20000          	|  0.10412        	|  0.0052227      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.1473e-06     	|  20000          	|  0.10295        	|  0.0051635      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  4.8254e-06     	|  20000          	|  0.096509       	|  0.0048407      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  4.6706e-06     	|  20000          	|  0.093413       	|  0.0046854      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  4.5519e-06     	|  20000          	|  0.091037       	|  0.0045662      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  4.4871e-06     	|  20000          	|  0.089742       	|  0.0045013      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.4682e-06     	|  20000          	|  0.089365       	|  0.0044824      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  4.4518e-06     	|  20000          	|  0.089036       	|  0.0044659      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  4.3054e-06     	|  20000          	|  0.086108       	|  0.004319       	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  4.3011e-06     	|  20000          	|  0.086022       	|  0.0043147      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  4.174e-06      	|  20000          	|  0.083481       	|  0.0041872      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  4.0485e-06     	|  20000          	|  0.080971       	|  0.0040613      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  4.0428e-06     	|  20000          	|  0.080856       	|  0.0040556      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  4.0333e-06     	|  20000          	|  0.080665       	|  0.004046       	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  4.022e-06      	|  20000          	|  0.080439       	|  0.0040347      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  4.0206e-06     	|  20000          	|  0.080411       	|  0.0040333      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  3.9597e-06     	|  20000          	|  0.079194       	|  0.0039722      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  3.948e-06      	|  20000          	|  0.078961       	|  0.0039605      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  3.945e-06      	|  20000          	|  0.0789         	|  0.0039575      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  3.9436e-06     	|  20000          	|  0.078872       	|  0.0039561      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  3.9414e-06     	|  20000          	|  0.078829       	|  0.0039539      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.037987       	|  2              	|  0.075974       	|  0.0038107      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.7301e-06     	|  20310          	|  0.075758       	|  0.0037999      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  3.2153e-06     	|  20310          	|  0.065303       	|  0.0032755      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  2.5611e-06     	|  20000          	|  0.051222       	|  0.0025692      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  2.4544e-06     	|  20000          	|  0.049088       	|  0.0024622      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  2.4424e-06     	|  20000          	|  0.048848       	|  0.0024501      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  2.3527e-06     	|  20000          	|  0.047053       	|  0.0023601      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  2.2812e-06     	|  20000          	|  0.045624       	|  0.0022884      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  2.2537e-06     	|  20000          	|  0.045074       	|  0.0022608      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  2.2214e-06     	|  20000          	|  0.044428       	|  0.0022284      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  2.2139e-06     	|  20000          	|  0.044277       	|  0.0022209      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  2.1891e-06     	|  20000          	|  0.043782       	|  0.002196       	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00055477     	|  20             	|  0.011095       	|  0.00055653     	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0049175      	|  2              	|  0.009835       	|  0.00049331     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00027161     	|  20             	|  0.0054322      	|  0.00027247     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  1.6379e-05     	|  310            	|  0.0050775      	|  0.00025468     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0047165      	|  1              	|  0.0047165      	|  0.00023657     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00014012     	|  20             	|  0.0028023      	|  0.00014056     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  8.0475e-06     	|  310            	|  0.0024947      	|  0.00012513     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  5.0312e-06     	|  310            	|  0.0015597      	|  7.823e-05      	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00075864     	|  2              	|  0.0015173      	|  7.6104e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  4.8059e-06     	|  310            	|  0.0014898      	|  7.4727e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  4.6768e-06     	|  310            	|  0.0014498      	|  7.272e-05      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  4.4495e-06     	|  310            	|  0.0013793      	|  6.9185e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  3.8266e-06     	|  310            	|  0.0011862      	|  5.9499e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00056798     	|  2              	|  0.001136       	|  5.6977e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  3.509e-06      	|  310            	|  0.0010878      	|  5.4561e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  3.0574e-06     	|  310            	|  0.0009478      	|  4.754e-05      	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00043004     	|  2              	|  0.00086008     	|  4.314e-05      	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00085827     	|  1              	|  0.00085827     	|  4.3049e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.0003236      	|  2              	|  0.00064721     	|  3.2463e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00060607     	|  1              	|  0.00060607     	|  3.0399e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00042484     	|  1              	|  0.00042484     	|  2.1309e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.000289       	|  1              	|  0.000289       	|  1.4496e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  7.1935e-06     	|  20             	|  0.00014387     	|  7.2162e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  6.104e-06      	|  20             	|  0.00012208     	|  6.1233e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  4.3649e-06     	|  22             	|  9.6027e-05     	|  4.8165e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  4.3617e-06     	|  22             	|  9.5957e-05     	|  4.813e-06      	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  4.0348e-06     	|  22             	|  8.8766e-05     	|  4.4523e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  3.9905e-06     	|  22             	|  8.7791e-05     	|  4.4034e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  3.8951e-06     	|  22             	|  8.5692e-05     	|  4.2981e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  3.8761e-06     	|  22             	|  8.5274e-05     	|  4.2772e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  4.0891e-05     	|  2              	|  8.1782e-05     	|  4.102e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  4.0786e-06     	|  20             	|  8.1572e-05     	|  4.0915e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  3.0127e-06     	|  22             	|  6.628e-05      	|  3.3245e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  2.8601e-06     	|  20             	|  5.7202e-05     	|  2.8691e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  2.7344e-06     	|  20             	|  5.4688e-05     	|  2.7431e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  2.4286e-06     	|  22             	|  5.343e-05      	|  2.6799e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.9237e-05     	|  1              	|  4.9237e-05     	|  2.4696e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  9.7081e-06     	|  2              	|  1.9416e-05     	|  9.7388e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  7.5432e-06     	|  2              	|  1.5086e-05     	|  7.5671e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  6.5994e-06     	|  2              	|  1.3199e-05     	|  6.6202e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  1.2292e-05     	|  1              	|  1.2292e-05     	|  6.1652e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  6.1458e-06     	|  2              	|  1.2292e-05     	|  6.1652e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  5.8319e-06     	|  2              	|  1.1664e-05     	|  5.8504e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.0476e-05     	|  1              	|  1.0476e-05     	|  5.2543e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.0407e-05     	|  1              	|  1.0407e-05     	|  5.2197e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  4.7837e-06     	|  2              	|  9.5675e-06     	|  4.7989e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  4.7497e-06     	|  2              	|  9.4995e-06     	|  4.7648e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  4.5043e-06     	|  2              	|  9.0087e-06     	|  4.5186e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  8.9398e-06     	|  1              	|  8.9398e-06     	|  4.484e-07      	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  8.0308e-06     	|  1              	|  8.0308e-06     	|  4.0281e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  7.6126e-06     	|  1              	|  7.6126e-06     	|  3.8183e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.8063e-06     	|  2              	|  7.6126e-06     	|  3.8183e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  3.8063e-06     	|  2              	|  7.6126e-06     	|  3.8183e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  7.5437e-06     	|  1              	|  7.5437e-06     	|  3.7838e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  7.5428e-06     	|  1              	|  7.5428e-06     	|  3.7833e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  3.5274e-06     	|  2              	|  7.0548e-06     	|  3.5385e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  3.2829e-06     	|  2              	|  6.5658e-06     	|  3.2933e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  6.2156e-06     	|  1              	|  6.2156e-06     	|  3.1176e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  3.0826e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  3.0826e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  6.1458e-06     	|  1              	|  6.1458e-06     	|  3.0826e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  5.6578e-06     	|  1              	|  5.6578e-06     	|  2.8378e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.8289e-06     	|  2              	|  5.6578e-06     	|  2.8378e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  5.6578e-06     	|  1              	|  5.6578e-06     	|  2.8378e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  2.8374e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  2.8374e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  2.8374e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  5.587e-06      	|  1              	|  5.587e-06      	|  2.8023e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.587e-06      	|  1              	|  5.587e-06      	|  2.8023e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  5.1679e-06     	|  1              	|  5.1679e-06     	|  2.5921e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  4.7507e-06     	|  1              	|  4.7507e-06     	|  2.3828e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  4.3297e-06     	|  1              	|  4.3297e-06     	|  2.1717e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.2599e-06     	|  1              	|  4.2599e-06     	|  2.1367e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  4.191e-06      	|  1              	|  4.191e-06      	|  2.1021e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  4.19e-06       	|  1              	|  4.19e-06       	|  2.1016e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  1.9269e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.7728e-06     	|  1              	|  3.7728e-06     	|  1.8924e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.8919e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.8919e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.8919e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.8914e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.8914e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.8914e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.702e-06      	|  1              	|  3.702e-06      	|  1.8569e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  3.702e-06      	|  1              	|  3.702e-06      	|  1.8569e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.702e-06      	|  1              	|  3.702e-06      	|  1.8569e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.7011e-06     	|  1              	|  3.7011e-06     	|  1.8564e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.8161e-06     	|  2              	|  3.6322e-06     	|  1.8218e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.3537e-06     	|  1              	|  3.3537e-06     	|  1.6821e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.3518e-06     	|  1              	|  3.3518e-06     	|  1.6812e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  2.3749e-06     	|  1              	|  2.3749e-06     	|  1.1912e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
