- Arrangement of some elements according to some
*total order*:- Total:
∀x,y x≤y∨y≤x - Reflexive:
x≤x - Antisymmetric:
x≤y∧y≤x⟺x=y - Transitive:
x≤y∧y≤z⟹x≤z

- Total:
- Inversion: pair of elements out of order
- Sorting goal is to make number of inversions zero

- Find smallest item
- Swap this item to the front and ‘fix’ it
- Repeat for unfixed items until all items are fixed
- Time complexity:
Θ(N2) - Inefficient

- Naive heapsort:
- Insert all items into a max heap, and discard input array. Create output array
- Repeat N times:
- Delete largest item from the max heap.
- Put largest item at the end of the unused part of the output array.

- Time complexity:
Θ(NlogN) time to put all items in heapΘ(1) time to get largest item, doneN timesΘ(NlogN) time to do allN removals from heap2⋅Θ(NlogN)+Θ(N)=Θ(NlogN) runtime!

- Memory usage is
Θ(N) to build additional copy of all data

- In-place heapsort:
- Convert array into heap via
*"bottom-up heapification"*(make array itself a heap)- Sink nodes in heap (move them down the heap) in reverse level order (reverse Breath-First-Search order, lower rows sunk first, and higher rows last)
- Still
Θ(NlogN) time complexity, but nowΘ(1) space complexity because no additional memory used- Bottom-up heapification is
Θ(N) in the worst case Θ(1) space complexity is only in iterative heap operations; if we have recursive heap operations then to keep track of frames we haveΘ(logN) space complexity (very minor)

- Bottom-up heapification is

- Convert array into heap via

- Recursively split an input item into two halves
- These two halves are further split until a base case of size one

- Recursively combine smaller items into a bigger item
- Combine until back at item of size of input array
- Combine by having two pointers, one for each of the two smaller sorted items (starting from base case of size one item, which is sorted)
- We "step" through the two items with pointers and compare values to construct a larger sorted item: pointers start at the first element in respective items, and compare values; smaller item is put in the larger to-return item, and corresponding pointer is advanced
- Done until all elements are transfered to the larger item

- Time complexity:
Θ(NlogN) , space complexity:Θ(N) - Faster than heap sort!

- Naive approach:
- Start with an empty output array
- For each element in the input, walk through the current output array and find the element's correct position, and insert it there
- Repeat until all input elements have been inserted into output

- More efficient approach:
- Instead of inserting and needing to move elements over, do in-place swapping
- We have a 'sorted' and 'unsorted' section of our array; keep track of this division via pointers (originally pointer at first element, so all in unsorted part and none in sorted)
- Let's say we have
X items in the sorted partition of the array, and a remainingY items in the unsorted partition to go- Find the first item currently in unsorted, from the pointer we use to keep track of this partition
- Advance this pointer, moving this first item of
Y to the sorted position. With a*second*pointer, walk backwards through the array (from the partition index to the array's start), pairwise swapping the newly inserted item with the elements that already were in the sorted part of the array - Keep doing these pairwise swaps as long as the item at the second pointer is larger than the item we newly added to the sorted part of the array. Each time we swap two neighboring items in the array, we advance the pointer by decreasing it's index by one (moving it closer to the start of the array).
- When the item at the second pointer is smaller or equal to the newly added item, we are done swapping and have properly inserted the new element into the sorted section
- Repeat till all elements are in the sorted array (first pointer reaches end of array)

- Time complexity:
Ω(N) best case, andO(N2) worst case- Insertion sort is better than merge sort and the others for a small
N , or an input array that is*almost sorted* - Almost sorted:
- For an array with only
K inversions, insertion sort has a runtime ofΘ(N+K) - An array is almost sorted if the number of inversions
K is≤cN for some constantc

- For an array with only
- For small
N (≈N≤15 ), insertion sort is faster than all the other algorithms Θ(1) space complexity for in-place sort