Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speedup compressed features loading #20

Merged
merged 1 commit into from
Sep 27, 2018

Conversation

bobchennan
Copy link
Contributor

Now it is much faster for larger data and I also tested with TEST_COMPRESSED_FEATS.py:

Before:

0.0351510047913
0.0127110481262
(u'AMI_ES2011a_H00_FEE041_0005553_0005685', 0.0012089051)
(u'AMI_ES2011a_H00_FEE041_0003714_0003915', 0.006009698)
(u'AMI_ES2011a_H00_FEE041_0004643_0004763', 0.0013620704)
(u'AMI_ES2011a_H00_FEE041_0003427_0003714', 0.006772384)
(u'AMI_ES2011a_H00_FEE041_0005133_0005553', 0.0052853823)
(u'AMI_ES2011a_H00_FEE041_0004763_0005020', 0.0033664554)
(u'AMI_ES2011a_H00_FEE041_0005856_0006217', 0.006143585)
(u'AMI_ES2011a_H00_FEE041_0004332_0004439', 0.0022902265)
(u'AMI_ES2011a_H00_FEE041_0005020_0005133', 0.002434183)
(u'AMI_ES2011a_H00_FEE041_0003915_0004332', 0.0066184625)

After:

0.00288486480713
0.00546503067017
(u'AMI_ES2011a_H00_FEE041_0005553_0005685', 0.0016530156)
(u'AMI_ES2011a_H00_FEE041_0003714_0003915', 0.004995942)
(u'AMI_ES2011a_H00_FEE041_0004643_0004763', 0.0010585152)
(u'AMI_ES2011a_H00_FEE041_0003427_0003714', 0.005950749)
(u'AMI_ES2011a_H00_FEE041_0005133_0005553', 0.0082992315)
(u'AMI_ES2011a_H00_FEE041_0004763_0005020', 0.0045164824)
(u'AMI_ES2011a_H00_FEE041_0005856_0006217', 0.0057396963)
(u'AMI_ES2011a_H00_FEE041_0004332_0004439', 0.0020962954)
(u'AMI_ES2011a_H00_FEE041_0005020_0005133', 0.0034917593)
(u'AMI_ES2011a_H00_FEE041_0003915_0004332', 0.0073646307)

Running time is not meaningful since it is too small. It is still not identical but errors are in similar range.

@KarelVesely84
Copy link
Owner

Hi, it looks good. Aha, I see it is faster because it is not done 'element-wise' but 'matrix-wise'.
Did you check if it runs both with 'python2' and 'python3' ? If yes, I'll merge it...

@KarelVesely84
Copy link
Owner

And thank you for creating the change, the coffee spirit of the Kaldi community will be grateful :)

@bobchennan
Copy link
Contributor Author

Yes I tested with python2.7 and python3.5.3.

Running log for python 2.7:

0.0194351673126
0.00395321846008
(u'AMI_ES2011a_H00_FEE041_0005553_0005685', 0.0016530156)
(u'AMI_ES2011a_H00_FEE041_0003714_0003915', 0.004995942)
(u'AMI_ES2011a_H00_FEE041_0004643_0004763', 0.0010585152)
(u'AMI_ES2011a_H00_FEE041_0003427_0003714', 0.005950749)
(u'AMI_ES2011a_H00_FEE041_0005133_0005553', 0.0082992315)
(u'AMI_ES2011a_H00_FEE041_0004763_0005020', 0.0045164824)
(u'AMI_ES2011a_H00_FEE041_0005856_0006217', 0.0057396963)
(u'AMI_ES2011a_H00_FEE041_0004332_0004439', 0.0020962954)
(u'AMI_ES2011a_H00_FEE041_0005020_0005133', 0.0034917593)
(u'AMI_ES2011a_H00_FEE041_0003915_0004332', 0.0073646307)

Running log for python 3.5.3:

0.0035649039782583714
0.006509726983495057
AMI_ES2011a_H00_FEE041_0003427_0003714 0.005950749
AMI_ES2011a_H00_FEE041_0004643_0004763 0.0010585152
AMI_ES2011a_H00_FEE041_0004763_0005020 0.0045164824
AMI_ES2011a_H00_FEE041_0004332_0004439 0.0020962954
AMI_ES2011a_H00_FEE041_0003915_0004332 0.0073646307
AMI_ES2011a_H00_FEE041_0005553_0005685 0.0016530156
AMI_ES2011a_H00_FEE041_0005133_0005553 0.0082992315
AMI_ES2011a_H00_FEE041_0003714_0003915 0.004995942
AMI_ES2011a_H00_FEE041_0005020_0005133 0.0034917593
AMI_ES2011a_H00_FEE041_0005856_0006217 0.0057396963

@KarelVesely84
Copy link
Owner

okay, merging!

@KarelVesely84 KarelVesely84 merged commit adadad1 into KarelVesely84:master Sep 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants