http://www.geocities.com/hirak_99/goodies/finddups.html

So here is a copy of the zip file:

finddups

There are a number of alternatives in this question and on alternative.to, but I’m not sure if they meet my needs yet.  Since this still works in Wine (with a dll from a real installation of Windows) and Windows 7, I’ll probably continue to use it.

Strengths:

• Colors clearly separate groups of identical files
• Arranged in order of size, so you can cancel the search, delete a few files, and clear up lots of space, without wasting time on small files
• It won’t delete all copies of a file unless you explicitly tell it to.  It always keeps at least one.
• It allows you to select a directory, so that all files in that directory or subdirectories will be marked for deletion (unless that would delete all copies of a file).
• If it deletes all files in a directory, it can be set to delete the directory itself, too, so you can clean out an entire directory tree at once, automatically, without accidentally losing anything unique

Weaknesses:

• Written a long time ago
• Doesn’t look right with high-res screens.
• Can’t read files with Unicode filenames
• In Wine, it sees links as regular files, and will delete an actual file while sparing the link.

• Several different methods of estimating the frequency of a waveform
• Peak detection for spectra
• Measure total harmonic distortion plus noise of a sine wave (THD+N)
• Apply an A-weighting filter to audio files

I’m sort of intending to combine them all into one eventually.

• Outputs sine, pink noise, and white noise
• Volume and frequency sliders
  • Controls are hard to grab at the edges
  • Volume control is independent from the phone’s volume control
  • Doesn’t go below 100 Hz
• Sine waves are distorted and badly aliased at high frequencies.  THD+N measurements:
  • 100 Hz 0.149%
  • 1 kHz: 0.08%
  • 10 kHz: 7.1%
  • 20 kHz: 20.15%
  • Spectrum for 997 Hz:
  • 
• Although the control says “1.00 kHz” at startup, it’s actually playing 765 Hz.  Frequencies are correct after you start moving it around, though.
• White noise level is higher than sine wave — sine wave never reaches the peak output of the phone, even at “0 dB”
• There doesn’t seem to be any way to enter levels other than 0 dB manually, since it doesn’t let you type a minus sign.
• White noise sounds like it’s repeating every 1.4 seconds
• Noise is the same in both channels

Frequency generator 200909150

• Outputs sine, square, triangle, sawtooth
• Playing more than one sine wave causes clipping, but you can decrease the phone’s volume control
• Default setting is 440, 444, and 448 Hz, but when played together, the waveform changes abruptly once per second
• Sine wave THD+N:
  • 40 Hz: 1.00%
  • 440 Hz: 0.056%
  • 10 kHz: 0.051%
  • 20 kHz: 0.222%
• I don’t know how the square, triangle and sawtooth are generated, but it ain’t right.
  • This is what the “square wave” looks like at 10 kHz:
  • 
  • And this is the spectrum:
  • 
  • Nooot even close

(Using ExtUSB headphone cable, Adobe Audition, Audio Precision.)

In normal people words, ICA is a form of blind source separation — a method of unmixing signals after they have been mixed together, without knowing exactly how they were mixed. It’s not as bad as Wikipedia makes it sound. It’s just the signal processing equivalent of this:

One of the problems I always have with learning stuff like this is the lack of clear examples. They exist, but they’re not generally very good. (And why do researchers always work with awful noisy 3-second 8 kHz recordings?) So, upon getting working results, I wrote up this little example.  This is in Python and requires the MDP (python-mdp in Ubuntu) and Audiolab packages (sudo easy_install scikits.audiolab).

In order for ICA to work, it requires at least one different recording for each signal you want to unmix. So if you have two musical instruments playing together in a room, and want to unmix them to get separate recordings of each individual instrument, you’ll need two different recordings of the mixture to work with (like a stereo microphone). If you have three instruments playing together, you’ll need three microphones to separate out all three original signals, etc. So, first, create the mix:

1. Find or make two mono sound files. I just used clips of music.
2. Mix them together to a stereo track, with both sounds mixed into both channels, but with each panned a little differently, so the two channels are not identical. They should sound all jumbled together, but the left channel should sound slightly different from the right.
3. Save in a format that libsndfile can read, like FLAC or WAV (not mp3):
   • Mixed music

So now you have the mixed signals, and you can pretend you don’t know how they were mixed. To unmix them automatically, run something like this in Python:

from mdp import fastica
from scikits.audiolab import flacread, flacwrite
from numpy import abs, max

# Load in the stereo file
recording, fs, enc = flacread('mix.flac')

# Perform FastICA algorithm on the two channels
sources = fastica(recording)

# The output levels of this algorithm are arbitrary, so normalize them to 1.0.
sources /= max(abs(sources), axis = 0)

# Write back to a file
flacwrite(sources, 'sources.flac', fs, enc)

The output has each signal in its own channel:

• Demixed music

You can hear some crosstalk, but it’s pretty good:

For more than two sources, I just read them in separately and combined them in Python:

rec1, fs, enc = flacread('Mixdown (1).flac') # Mono file
rec2, fs, enc = flacread('Mixdown (2).flac')
rec3, fs, enc = flacread('Mixdown (3).flac')

sources = fastica(array([rec1,rec2,rec3]).transpose())

flacwrite() has no problem writing multi-channel files.

Mixed speech:

After demixing, there’s very little crosstalk, though the noise floor increases considerably.  This seems to be the case when the mixes are very similar:

Although this method was recommended to me for real-life audio signals and microphones, as I’ve described above, it turns out that ICA doesn’t actually work well when the signals occur at different delays in the different sensor channels; it assumes instantaneous mixing (that the signals are in perfect sync with each other in all the different recordings).  Delay would happen in a real-life situation with performers and microphones, since each source is a different distance from each microphone. This is exactly the application I had in mind, though, so I don’t really have any further interest in ICA…

These charts lump together upgrades and clean installs, since the ratios are similar for both, and it shows more clearly the overall trend.

• The green bar is for the percent of users who found the release “worked flawlessly”.
• Yellow is for “few things to fix, nothing serious though”.
• Red is for “many problems that I’ve not been able to solve”.

With the release of Karmic, I see a small trend of an increasing green bar, which is great, but the releases aren’t terribly different.  I’d like to see the red bar get drastically shorter with each release.  I guess that’s not as high of a priority…

• Again, the language selection screen is ugly.
  • 
• Again, you have to reboot after checking the CD for errors.
• Currently it says “Press F4 to select alternative start-up and installation modes“, but it should explicitly say “Press F1 for help.”, first, for ultimate newcomers.
  • 
• After pressing F1, Help page should describe basics at top, for ultimate newcomers.  “Use arrow keys to move, press Enter to select etc”. Right now it just looks like this:

  •  HELP INDEX 
    
     KEY    TOPIC 
    
    <F1>   This page, the help index.
    <F2>   Prerequisites for running Ubuntu.
    <F3>   Boot methods for special ways of using this CD-ROM.
    <F4>   Additional boot methods; rescuing a broken system.
    <F5>   Special boot parameters, overview.
    <F6>   Special boot parameters for special machines.
    <F7>   Special boot parameters for selected disk controllers.
    <F8>   Special boot parameters for the bootstrap system.
    <F9>   How to get help.
    <F10>  Copyrights and warranties.

  • 
• Release note complaints still apply.
  • It should show a rich text box with scroll bars that contains the most recent release notes, and offer to download the rest if you have a connection. If there is a connection, download the latest rich text release notes and display them automatically.  If not, display the ones packaged with the CD, and a note saying they are out of date.
  • “If you have Internet access, read the release notes for information on problems that may affect you.”    (Bug 254639)
  • 
  • Why in the world does it open up Firefox when it could just display the release notes in a rich text box in the installer itself?  There’s plenty of room.
  • 
  • Should be able to prompt for a wireless key
• It says “Select your time zone from the map”, but you’re really selecting location, not just time zone.  Does this affect language, keyboard layout, weather, units, or other local properties?  If not, it probably should.  (Bug 380171)
  • 
• Partitioner is the worst part:
  • “Install them side by side” needs wording improvement.  What is “them”?
  • It’s good that it warns “This will delete x and install y“
  • Don’t show a drop-down box for disk selection if there is only one disk  (Bug 380175)
    • Use a list (with dynamic scrollbar if needed) instead so they can see and understand instantly that they have multiple choices; plenty of screen real estate for this
  • If you have multiple disks, it only shows a color bar for one disk at a time.  This drop-down selector is for that, too?  If so, shouldn’t it be at the top, instead of inside an option so that it looks like it only applies to that option?
  • If there is a big empty space on a drive, default to that. If the disk is full of partitions, but one is Linux format and completely empty, default to that.
    • This isn’t just a matter of what option comes up as default, it’s also a matter of the slider not stopping at partition boundaries.
    • So if you try to use an empty space, you’ll end up resizing an existing partition because the graphical slider mechanism only gives you so much accuracy.
    • It should just default to the boundaries of the empty space if possible, and the slider should stop at partition boundaries.
    • “Use the largest continuous free space” doesn’t even work.
  • If there are human-readable descriptions of disks and partitions, show those first.  Showing /dev/whatever should be parenthetical.
  • Why does it show a brown bar if I select manual install?  What is brown supposed to signify?
  • Slider behaves super oddly, moving all above partitions at once as I slide it.
  • This partitioner scares me, so i canceled it and modified the partitions in GParted first, then went back into the installer and used manual mode to mount the partition I wanted as root.
    • I’d like to see Canonical take a survey about the partitioner.  Does anyone actually use it?  Do most just use manual mode?  Does it really only work on drives that have Windows and nothing else on them?
  • Ask if I want to shrink another partition?
  • Partitions are not obvious.  Brown and a different shade of brown right next to each other.
  • Should show used and free space for each partition.  This helps in identifying them, helps in seeing how much space there is to work with when resizing them.
  • Should have same functions as GParted, basically (Idea 19218)
• “What is the name of this computer?” prompt:  Are there any recommendations on what this should be?  Should it be all lowercase or less than a certain length for compatibility?  If so, it should say so.
• In step 6, it says “Migrate documents and settings”, which presumably copies settings from existing partitions?  It didn’t recognize my Windows partition or my Ubuntu partition for migration.  It should do both.
• It should show the password strength meter when entering the main user’s password for the first time. (Bug 155308)  Apparently it does show a “weak password” dialog if you type less than 8 characters, though.
• The boot noise infuriates me.  I’ve heard it so many times now, and there’s no way to shut it off.  If you boot to the LiveCD, there’s going to be an extremely loud drum sound on startup, and you can’t do anything about it.  The volume buttons don’t kick in until after the sound has played.  I tried to leave the room whenever I knew it was going to happen, or cover the laptop with pillows.  (Bug 114160)
• After finishing the installation, my computer would no longer boot.  Bad Ubuntu.  It should warn the user before installing in a location that’s not accessible by the BIOS.  I had to create a separate boot partition to get it to work.  (Bug 379348)

• Language selection screen after booting from CD is jarring, not totally intuitive, ugly. Fills the entire screen with no explanation of what you are supposed to do with it, though this might seem obvious.
  
• After checking the CD for errors, why does it need to reboot again?  It should just go back to the CD menu.
  • 
• Progress bar during startup of LiveCD should have some explanatory text about what it’s doing (”Please wait — loading the Ubuntu desktop”)
• Startup sound can be pretty startling if the sound is turned all the way up by default.  The sound file itself should not start with a sudden onset of sound, but be a gradual increase.   Then, even if it’s loud, you would at least have some warning.
• “If you have Internet access, read the release notes for information on problems that may affect you.”  What a stupid message.
  • It should be able to tell automatically if you have Internet access.
  • If you don’t have Internet access, then there’s no way for you to know that you are going to have problems. The release notes should be installed on the CD itself, and it should show the date at which they were last updated.
  • If you do have Internet access, it should download the latest release notes automatically and show them to you.
  • LP #254639: Installer doesn’t know whether you have Internet access
• Keyboard layout selection should include images of keyboard layouts, and a better explanation of what a keyboard layout actually is.
• “How do you want to partition the disk” isn’t newb-friendly.  Explain what it means to partition a disk and why you might want to do it differently.
  • “Guided” isn’t very explanatory.
  • 
• System beeps are scary.  They occur throughout the installation and even after the installation.  Why?
• Icons and button icons should follow color scheme.  If you change the selection color, the title bars change color, but the folder icons are still orange.  (If icons are SVG, could this be done with CSS-based colors?)
  • UB #5238: Simple way to change the Human theme base color
• While waiting for things to install, looked through the Examples directory.  Howard County library PDF has a typo: “central Marylan”
  • LP #223382: Embarrassing spelling error in hardy example-content
• Examples could use a major revamp.
• Another document in the Examples folder says to click on a document to open it.  This is dumb because:
  • They have to do that to open the document they are reading in the first place.
  • It’s a double-click by default.
• We really need a beginner’s tutorial.
  • Tutorial/intro should be purely optional, easy to kill, and not get in the way, but it should be obvious and discoverable, too.
    • Notification area balloon that pops up each time you log in until you click it and choose to either go through the tutorial or tell it to shut up.
    • Icon on the Desktop that you can delete
  • Maybe include a downloadable video tutorial, too?  Won’t fit on the CD unless you use a format specially made for screencasts?  Animated GIF?
  • Don’t give false expectations:  “Ubuntu has a wealth of programs to choose from – over 19,000 at last count!”  Most of these are just components of other programs, and many are useless garbage.  Be realistic and then new users won’t be so pissed off when things don’t work.  Sell Ubuntu by making Ubuntu better, not by lying about what it’s capable of.
  • “Ubuntu is free of charge and open source, and is created and maintained by volunteer/Canonical employees.  If you’re having trouble, remember that there’s probably a way to do what you want, even if it’s not the way you’re used to.   For instance, instead of browsing the web with Internet Explorer, Ubuntu comes with Firefox web browser, which is stable, extensible, blah blah.  (You can install Firefox in Windows and OS X too, but in Ubuntu it’s the default browser.)  Instead of Microsoft Office, try OpenOffice or GnomeOffice, they’re under the “Office” tab in the Applications menu.  OpenOffice has much of the same functionality of MS Office, and can open your old files, too.  Instead of downloading .EXEs to install programs, we download all packages from the Ubuntu repositories, which ensures that it has been thoroughly tested and built specifically for Ubuntu...”
  • Maybe just make one of the Examples a PDF with a table of common actions in Windows, Mac OS X, and Ubuntu, both as a cheatsheet newcomer reference and to drive home the point that Ubuntu is not Windows; it is a different OS, just like OS X is a different OS from Windows.
• Openoffice splash screen takes too long to load after double click.  Appears like nothing has happened.  Firefox has the same problem.  Always should have some kind of feedback.
• After upgrading to a new kernel and booting into it x times, it should assume that everything’s working fine and pop up a notification balloon explaining what a kernel is and telling you it’s probably ok to remove the old one and this is how you do it.  Won’t appear again for that kernel, and can be easily turned off permanently for those people who are infuriated whenever the computer offers a helpful message.
• When installing restricted drivers in Jockey
  • “In order to use your hardware more efficiently, you can enable drivers which are not free software.”
    • Let’s be realistic.  99% of users don’t care a bit whether they have the freedom to modify and redistribute their video drivers.  Most don’t even know how to write a bash script (and we should all be working towards a world where they’ll never have to)
  • There’s a long pause where it says “downloading and installing”, but the progress bar just sits there at 0%, appearing to be broken.  Feedback, please!
  • Jockey gives stupid unnecessarily scary message about how proprietary drivers from Ubuntu’s repositories “represent a risk to you”, as if they are somehow malicious or more buggy than the Free ones.  (Yeah, right.)   LP# 381805
  • Does the sideways screen widening attack thing, but that’s been fixed.
    • LP #278071: jockey-gtk weird installed driver display
• Flash installer is cryptic
  • “Choose a plugin” provides three possibilities, but no explanation of what they are, what the benefits of each are, or what anything means.  (Screenshot)
  • “No description found in plugin database”.   Great.  Thanks.
  • Then it asks to install it again if I go to another tab that has Flash content without restarting first.
• Sound themes:
  • should be a link to download more sound themes
  • When browsing for sound files, it’s not obvious where the sound themes are stored.  There is actually a link on the bottom left, which is helpful, but it doesn’t go there by default and is not obvious at the bottom of a list of drives.
  • Instead of a file chooser window, there should  be a dedicated System Sounds Chooser.  If you want a sound that’s not provided by that, then you can use a file chooser to select it.   But all the theme sounds should be available from a simple app with a previewer/player.
• Fonts are too small and seem strangely fuzzy by default — weird, since they were working fine in my previous install.  Maybe settings are different.
• Scrolling in Firefox is jerky no matter how I set up Compiz or turn it off.  (There are so many bugs filed about this I don’t even know which is the culprit and I don’t care to pursue it.)
• Title bars keep screwing up, probably just Compiz.
• Backspace in Firefox does nothing.
  • (Yeah right — like that’s ever going to be fixed.)
  • LP #60995: The backspace key pages up instead of going back in history
• Touchpad tapping is enabled by default, and does not seem to turn off while typing, which causes me to click things by accident.

I should also have written down the stuff that I was impressed with, though, since there actually was a lot.  “Hey, that actually works out of the box now?  Nice!” Maybe I’ll do it again with Lubi or VirtualBox and keep more positive notes.

I used the More tag so that the link rel tag only triggers Ubiquity when you view the post itself, not when viewing an aggregate of several posts.

Or would that be a bad thing?

Here is a regular link:

test_command

Here is the link rel tag (which WordPress eats if you go back to Visual editing mode):

With the last few releases, though, I’ve felt that after “upgrading” my system, it actually gets worse. There are lots of bugs and regressions, and I wonder how this can possibly be an official “release”.  Intrepid was the worst so far: my video card stopped working, my keyboard’s function keys started locking up the screen, session saving doesn’t work, sound system is misconfigured, …  These are exactly the kind of headaches that led me to abandon Linux in the past and go back to Windows.  When I upgrade my computer, I expect the upgrade to fix problems, not cause more.

Is every Ubuntu release worse than the last?  To see if it’s just my own personal experience, I looked through the release polls on Ubuntu Forums and combined them all together into a little HTML percentage bar graph.  The green bar is for the percent of users who found the release “worked flawlessly”, after either an upgrade or clean install. Yellow is for “few things to fix, nothing serious though”. Red is for “many problems that I’ve not been able to solve”.  (The poll questions are the same for each release, and have 1,300 to 5,500 voters each.)

(Updated graphs including Jaunty)

It’s pretty clear that my experience is not unique:

Almost half of Intrepid users experienced “many problems” that they have been unable to solve.

You can see from the original results that the experiences for upgraders and first installers are roughly the same, proportionally.  (47% of Intrepid upgraders had serious problems, while 48% of first-timers had serious problems, for instance.) I’d speculate that this means that the main difference between people who have problems and people who don’t is the compatibility of their hardware.

I don’t know much about the technicalities, and haven’t followed the politics behind release cycles, but it seems that something needs to change in the priorities.  I don’t understand why a product would be shipped out the door as a “release”, when it has major bugs like this that were known in advance.  Isn’t that what betas and release candidates are for?

I’ve been told that I should wait a few months before upgrading, so that when I do, the bugs have been fixed.  But then… what’s the point of calling it an official release?  Why don’t you delay the release until those bugs are fixed?

I’ve been told that I should stick to LTS releases if I want fewer bugs, but I had plenty of problems with the Hardy upgrade, too.  The graph shows that Hardy was only marginally better.  Some even said the LTS release was “much more buggy than Gutsy or any other previous release“.

I’ve been told that Ubuntu’s release schedule is a response to Debian’s long delays between releases — that a rigid six-month schedule keeps programs like Firefox and OpenOffice up to date, but as a consequence we have to pay the price of a less stable system. I don’t understand why these are so intertwined.  I can download the latest bleeding-edge versions of software titles from getdeb.net and run them just fine on my system; why can’t the “flagship” titles be kept up-to-date, while the low-level subsystems are kept as stable as possible?  If I’m using the latest Firefox beta and it crashes, it’s no big deal, but if a less-technical user upgrades their computer and the video card or wireless card stops working, that’s a deal-breaker.

Also, why not keep the release candidates to a rigid schedule, but delay the actual release until the bugs are fixed? Users who just can’t wait for the upgrade can subscribe to the RCs instead of the actual releases, while normal users who want to use their computers instead of debug them can wait until the real release.  Maybe this is what’s supposed to happen with the betas and releases?  It’s not working.